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1. INTRODUCTION 


The U.S. TIMSS and TIMSS Advanced 1995 & 2015 Technical Report and User’s Guide provides an 
overview of the design and implementation of the Trends in International Mathematics and Science Study 
(TIMSS) 2015 and TIMSS Advanced 2015 in the United States and one participating benchmarking state: 
Florida. 


1.1 TIMSS and TIMSS Advanced 2015 in Brief 


The Trends in International Mathematics and Science Study (TIMSS) is an international comparative 
study designed to measure trends in mathematics and science achievement at the fourth and eighth grades, 
as well as to collect information about educational contexts (such as students’ schools, teachers, and 
homes) that may be related to student achievement. TIMSS has been administered every 4 years since 
1995, with the sixth and most recent administration, in 2015, providing a 20-year trend line. The United 
States has participated in every administration of TIMSS, which includes 1995, 1999, 2003, 2007, 2011, 
and 2015 for the eighth grade and all but 1999 for the fourth grade (when it was not administered 
internationally). TIMSS is designed to align broadly with mathematics and science curricula in the 
participating education systems and, therefore, to reflect students’ school-based learning. Because it is an 
international study, TIMSS provides valuable benchmarking information on how U.S. students compare 
to students around the world. 


TIMSS Advanced is an international comparative study designed to measure the advanced mathematics 
and physics achievement of students in their final year of high school who are taking or have taken 
advanced courses. TIMSS Advanced also collects information about educational contexts (such as 
schools and teachers) that may be related to advanced students’ achievement. TIMSS Advanced was 
administered previously, in 1995 and in 2008, and most recently in 2015. The United States participated 
in the 1995 and 2015 administrations. Like TIMSS, TIMSS Advanced is designed to align broadly with 
curricula in the participating education systems and, therefore, to reflect students’ school-based learning 
of advanced mathematics and physics. TIMSS Advanced can inform policymakers, researchers, 
educators, and the public about the degree to which students in the United States excel in advanced 
mathematics and physics and may be prepared to undertake more specialized study in science, 
technology, engineering, and mathematics compared to their international peers. 


TIMSS and TIMSS Advanced are both sponsored by the International Association for the Evaluation of 
Educational Achievement (IEA) and conducted, in the United States, by the National Center for 
Education Statistics (NCES) in the Institute of Education Sciences within the U.S. Department of 
Education. 


The TIMSS and TIMSS Advanced assessments are developed through an international collaborative 
process involving input from the U.S. and international experts in mathematics, science, and 
measurement. These experts develop frameworks that define the knowledge and skills to be assessed and 
are designed to align broadly with mathematics and science curricula in the participating countries. The 
results, therefore, suggest the degree to which students have learned mathematics and science concepts 
and skills likely to have been taught in school. 


A large and diverse group of education systems, spanning six of the world’s continents, participated in 
TIMSS 2015. These education systems included 49 IEA member countries and six benchmarking 
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participants‘ that participated in the fourth-grade assessment and 38 IEA member countries and six 
benchmarking participants that participated in the eighth-grade assessment. Nine education systems—all 
IEA member countries—participated in TIMSS Advanced 2015. 


A detailed explanation of TIMSS and TIMSS Advanced 2015 from an international perspective can be 
found in five reports published by the IEA and available online. 


a TIMSS 2015 International Results in Mathematics (Mullis et al. 2016a); 
4 TIMSS 2015 International Results in Science (Martin et al. 2016); 


a TIMSS Advanced 2015 International Results in Advanced Mathematics and Physics 
(Mullis et al. 2016b); 


7 Methods and Procedures in TIMSS 2015 (Martin, Mullis, and Hooper 2016a); and 


7 Methods and Procedures in TIMSS 2015 Advanced (Martin, Mullis, and Hooper 
2016b). 


AUS. national report is available as well. It is titled Highlights from TIMSS and TIMSS Advanced 2015: 
Mathematics and Science Achievement of U.S. Students in Grades 4 and 8 and in Advanced Courses at 
the End of High School in an International Context (Provasnik et al. 2016). This report describes the 
performance of U.S. students relative to their peers in other education systems, changes in mathematics 
and science achievement over time, and additional details about the achievement of U.S. students 
nationally and in the benchmarking states that are not available in the international reports. The report is 


available online at https://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2017002. 


1.2 TIMSS and TIMSS Advanced 2015 U.S. Data Collection 
Activities and Schedule 


Descriptions of data collection activities and schedule within the United States provide a foundation to 
understand the data. These activities are listed in exhibit 1-1 along with the timing of their 
implementation. The activities are described in detail in chapter 4 of this report. 


‘TEA member countries include both “countries” which are complete, independent political entities, and non-national entities (e.g., England, 
Hong Kong, or the Flemish community of Belgium). The international reports have slightly different participating countries and benchmarking 
participants, as there are different ways to count the countries. 
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Exhibit 1-1. Schedule for U.S. TIMSS and TIMSS Advanced 2015 data collection activities 


2014 2015 
Jan [Feb|Mar Apr May) Jun. Jul Aug|Sep_ Oct |Nov| Dec Jan |Feb|Mar| Apr|May| Jun | Jul |Aug|Sep| Oct 


TIMSS and TIMSS Advanced 2015 


School Sampling 


National 


State Benchmarking 


Hiring recruitment staff 


Recruitment phase 


Recruiting states 


Recruiting districts 


Recruiting schools 


Cultural adaptations made to items 


IEA approval of cultural adaptations 


Production of instruments (Westat & Pearson) 


Printing of instruments 


Printing of assessment booklets 


Printing of questionnaire booklets 


Distribution of materials to field staff/schools 


Scheduling Assessments 


Data Collection Staffing 
Hiring assessment supervisors 


Training for assessment supervisors 


Within School Sampling 


Class Listing Forms 


Student-Teacher Linkage Forms 


Tracking Forms 


Assessment sessions in schools 


Receipt control Scoring, Coding, Data Entry 


Document receipt and processing (Pearson) 


———————y 
—_—_—_—_—_ 
GVwVWVm—les_s=s 
os 
—————————— > 
@ 
oe» 
QRW””?:?:°:,...:: 
Instrumentation —————— 
—_—_ > 
@ 
o_o 
or 
oe 
oo 
@ 
—_—se 
o_—_—_-_ 
_—__ 
4 
—____ 
—_—_—_—_————— 
on» 
o_—_—____—_- 
-_—_ 
—_s 
es 
oe? 


Coding, editing, processing of data (Person) 


Constructed response scoring (Pearson) o~ 


Data File Preparation Geees=_ee_s=_ > 


Cleaning and preparation of datafiles (Westat) ea 
Submission of datafiles to the Data Processing Center e 


SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) and TIMSS Advanced, 2015. 


1.3 Overview of the Design and Administration of TIMSS and 
TIMSS Advanced 2015 


The basic parameters of the design and administration of TIMSS and TIMSS Advanced 2015 in the 
United States are outlined below. A more detailed explanation is provided in subsequent chapters of this 
report. Methods and Procedures in TIMSS 2015 (Martin, Mullis, and Hooper 2016a) and Methods and 
Procedures in TIMSS Advanced 2015 (Martin, Mullis, and Hooper 2016b) contain detail on the 
international design. 
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Sampling 


TIMSS and TIMSS Advanced are sample-based assessments, meaning that while only a sample of 
students take the assessments, they are selected in such a way as to allow the results to be generalizable to 
a larger target population. The TIMSS and TIMSS Advanced target populations are based on standardized 
definitions, and the sampling is conducted following standardized and refereed international procedures. 


TIMSS required participating countries and other education systems to draw probability samples of 
students who were nearing the end of their fourth or eighth year of formal schooling, counting from the 
first year of the International Standard Classification of Education (ISCED) Level 1 (or primary 
schooling). (For additional information on ISCED levels, see 
http://uis.unesco.org/en/topic/international-standard-classification-education-isced). In the United States, 
one sample was drawn to represent the nation at grade 4 and another at grade 8. The U.S. national sample 
included both public and private schools, randomly selected and weighted to be representative of the 
nation at grade 4 and at grade 8. (See the section on sampling weights and standard errors in this report 
for definitions.) In addition, because Florida participated in TIMSS 2015 as a benchmarking participant, 
separate public school samples were drawn for Florida at both grades. 


In total, the U.S. national sample in 2015 consisted of 250 schools and 10,029 students at grade 4, and 
246 schools and 10,221 students at grade 8. The weighted school response rate for the United States was 
77 percent at grade 4 before the use of substitute schools (schools substituted for originally sampled 
schools that refused to participate) and 85 percent with the inclusion of substitute schools. At grade 8, the 
weighted school response rate before the use of substitute schools was 78 percent and 84 percent with the 
inclusion of substitute schools. The weighted student response rate at grade 4 was 96 percent and at grade 
8 was 94 percent. Student response rates are based on a combined total of students from both sampled and 
substitute schools. 


TIMSS Advanced required participating countries to draw probability samples of students in their final 
year of secondary school—ISCED Level 3—who were taking or had taken courses in advanced 
mathematics or who were taking or had taken courses in physics. In the United States, two samples of 
twelfth-graders were drawn to represent the nation—one for advanced mathematics and one for physics. 
The courses that define the target populations had to cover most, if not all, of the advanced mathematics 
and physics topics that were outlined in the assessment frameworks. In the United States, this was defined 
as a calculus course for eligibility for the advanced mathematics population and a physics course, such as 
Advanced Placement (AP) physics, for the physics population. The U.S. national samples included both 
public and private schools, randomly selected and weighted to be representative of the nation’s advanced 
mathematics and physics students at the end of high school. (See the section on sampling weights and 
standard errors in this report for definitions.) 


In total, the U.S. national sample in 2015 consisted of 241 schools for advanced mathematics and 165 
schools for physics (of the original sample of 348 schools for both subjects). The weighted school 
response rate for the United States for advanced mathematics was 72 percent before the use of substitute 
schools and 76 percent with the inclusion of substitute schools. The weighted school response rate for the 
United States for physics was 65 percent before the use of substitute schools and 68 percent with the 
inclusion of substitute schools. In terms of the number of students, the U.S. national sample consisted of 
2,954 students in advanced mathematics and 2,932 students in physics. The weighted student response 
rate was 87 percent for advanced mathematics and 85 percent for physics. Student response rates are 
based on a combined total of students from both sampled and substitute schools. 
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Assessment Design 


TIMSS and TIMSS Advanced 2015 included student booklets containing assessment items as well as 
contextual background questionnaires. Questionnaires for principals and teachers were self-administered, 
primarily using an online survey system. The assessment booklets were constructed such that not all of 
the students responded to all of the items. This is consistent with other large-scale assessments, such as 
NAEP. To keep the testing burden to a minimum, and to ensure broad subject-matter coverage, TIMSS 
used a rotated block design that included both mathematics and science items for each student. That is, 
students encountered both mathematics and science items during the assessment. TIMSS Advanced also 
used a rotated block design, but each student took only one subject, either advanced mathematics or 
physics. 


The fourth-grade assessment consisted of 14 booklets, each requiring approximately 72 minutes. The 
eighth-grade assessment also consisted of 14 booklets, each requiring approximately 90 minutes. The 
assessments were given in two equally timed sections, with a 5- to 10-minute break in between. At both 
grades, the mathematics and science items were each assembled separately into 28 blocks of items. 
Booklets consisted of four blocks each, including at least one mathematics block and at least one science 
block. 


The TIMSS Advanced assessments consisted of six booklets for advanced mathematics and six booklets 
for physics, each requiring approximately 90 minutes. Each student completed just one booklet. In both 
subjects, booklets consisted of three blocks each. 


After the cognitive assessment, students completed a 30-minute questionnaire designed to provide 
information about their backgrounds, attitudes, and experiences in school. 


Test Administration 


Test administration for TIMSS and TIMSS Advanced in the United States started in April and continued 
through June 2015. The administration was carried out by professional staff trained according to the 
international guidelines. School personnel were asked only to assist with listings of students, the 
identification of space for testing in the school, and the specification of any parental consent procedures 
required. The International Study Center monitored compliance with the standardized procedures. 


Scoring 


Both the TIMSS and TIMSS Advanced assessment items included both multiple choice and constructed- 
response items. A scoring rubric (guide) was provided for every constructed response item. The National 
Research Coordinator in each country was responsible for the scoring and coding of data in that country, 
following established guidelines. 
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Scaling 


Total scores for mathematics and science in TIMSS, and advanced mathematics and physics in TIMSS 
Advanced, along with scores that reflect performance in specific domains of each subject, were estimated 
using an item response theory (IRT) model. For example, the TIMSS 2015 eighth-grade assessment had 
four scales describing four mathematics content areas and four science content areas, as well as three 
cognitive domains in each of mathematics and science. Benchmark scores were also derived. IRT 
estimation procedures were also used to place scores from the six TIMSS assessments conducted in 1995, 
1999, 2003, 2007, 2011, and 2015 on the same scale (the scale of the 1995 administration). This allows 
for the calculation of trends in achievement even though the makeup of the countries participating in 
TIMSS changes over time. Scaling for TIMSS Advanced follows a similar process, using data from the 
1995, 2008, and 2015 administrations. More detailed information can be found in the Methods and 
Procedures in TIMSS 2015 at http://timssandpirls.bc.edu/publications/timss/2015-methods.html and 
Methods and Procedures in TIMSS Advanced 2015 at http://timss.bc.edu/publications/timss/2015-a- 
methods.html. 


Plausible Values 


To keep student burden to a minimum, TIMSS and TIMSS Advanced purposefully administered a limited 
number of assessment items to each student—too few to produce accurate individual content-related scale 
scores for each student. The number of assessment items administered to each student, however, is 
sufficient to produce accurate group content-related scale scores for subgroups of the population. These 
scores are transformed during the scaling process into plausible values to characterize students 
participating in the assessment, given their background characteristics. Plausible values are imputed 
values and not test scores for individuals in the usual sense. If used individually, they provide biased 
estimates of the proficiencies of individual students. However, when grouped as intended, plausible 
values provide unbiased estimates of population characteristics (e.g., means and variances for groups). 


Plausible values represent what the performance of an individual on the entire assessment might have 
been, had it been observed. They are estimated as random draws (usually five) from an empirically 
derived distribution of score values based on the student’s observed responses to assessment items and on 
background variables. Each random draw from the distribution is considered a representative value from 
the distribution of potential scale scores for all students in the sample who have similar background 
characteristics and similar patterns of item responses. Differences between plausible values drawn for a 
single individual quantify the degree of error (the width of the spread) in the underlying distribution of 
possible scale scores that could have caused the observed performances. 


More detailed information can be found in the Methods and Procedures in TIMSS 2015 (Martin, Mullis, 
and Hooper 2016a). 


Weighting 


Responses from the groups of students were assigned sampling weights to adjust for the complex sample 
design that resulted in students having an unequal, but known, probability of selection. Additionally, an 
adjustment for school and student nonresponse was built into the weighting. The estimation of sampling 
weights was carried out by Statistics Canada. More detailed information can be found in the Methods and 
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Procedures in TIMSS 2015 (Martin, Mullis, and Hooper 2016a) and Methods and Procedures in TIMSS 
Advanced 2015 (Martin, Mullis, and Hooper 2016b). In analyses of the TIMSS and TIMSS Advanced 
data, it is necessary to use sampling weights to obtain accurate population estimates. 


1.4 Reporting TIMSS and TIMSS Advanced Results 


Achievement results from TIMSS and TIMSS Advanced are reported on a scale from 0 to 1,000, with a 
scale average of 500 and standard deviation of 100. Even though the education systems participating in 
TIMSS and TIMSS Advanced have changed across the assessment rounds from the first administration 
(TIMSS and TIMSS Advanced in 1995), comparisons between the 2015 results and prior results are still 
possible because the achievement scores in each of the assessments are placed on a scale that is not 
dependent on the list of participating education systems in any particular year. 


In addition to numerical scale results, TIMSS and TIMSS Advanced also include international 
benchmarks. The international benchmarks provide a way to interpret the scale scores and to understand 
how students’ proficiency in a subject varies at different points on the scales. Each successive point, or 
benchmark, is associated with the knowledge and skills that students successfully demonstrate at each 
level. TIMSS describes four benchmarks of achievement (Advanced, High, Intermediate, and Low) and 
TIMSS Advanced describes three benchmarks of achievement (Advanced, High, and Intermediate). 


A more detailed explanation of the assessment’s equating, scaling, and benchmarks can be found in the 
Methods and Procedures in TIMSS 2015 at http://timssandpirls.bc.edu/publications/timss/2015- 
methods.html and Methods and Procedures in TIMSS Advanced 2015 at 
http://timss.bc.edu/publications/timss/2015-a-methods.html. 


1.5 U.S. International and National Data Files 


Three versions of the U.S. national data are available as follows: 


a The TIMSS and TIMSS Advanced U.S. international data files that are available 
as part of the international database released by the International Study Center. The 
U.S.-specific TIMSS data files can be downloaded from 
http://timssandpirls.bc.edu/timss2015/international-database. The U.S.-specific 
TIMSS Advanced data files are available separately and can be downloaded from 
http://timssandpirls.bc.edu/timss2015/advanced-international-database. The TIMSS 
and TIMSS Advanced 2015 international data files are available in two versions: a 
public-use version and a restricted-use version. A number of variables are not 
included in the public-use version in order to minimize the risk of disclosing 
confidential information. These variables include information about students’ year and 
month of birth, testing date, as well as school enrollment (see exhibit 5-1 in chapter 5 
of this technical report for a complete list of variables removed from the international 
public-use data files). The restricted-use versions of the TIMSS and TIMSS Advanced 
2015 international data files are available by request through the IEA Study Data 
Repository (http://www.iea.nl/data.html) to obtain permission and access to the 
restricted-use version of the TIMSS and TIMSS Advanced 2015 international data 
files. A description of the restricted variables removed from the public-use data files 
are presented in chapter 4 of the TIMSS 2015 User Guide for the International 
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Database (Foy 2017a) and the TIMSS Advanced 2015 User Guide for the 
International Database (Foy 2017b). These data files conform to the international 
specifications common to the data files from all countries. However, they do not 
include the U.S.-specific adaptations made to a few questions in the questionnaires or 
the additional questions added to the school and student questionnaires, such as the 
question on race/ethnicity added to the student questionnaire. Data for Florida are not 
currently included in the international data files due to potential confidentiality issues. 
The Florida student file that meets confidentiality standards is expected to be released 
at a later date. 


7 The TIMSS and TIMSS Advanced U.S. national public-use data files that are 
available through the National Center for Education Statistics. The TIMSS and 
TIMSS Advanced U.S. national dataset can be downloaded from 
http://nces.ed.gov/timss/datafiles.asp. These U.S. data files include the U.S.-specific 
adaptations made to questionnaire items and additional questions added to the school 
and student questionnaires. The TIMSS and TIMSS Advanced U.S. national public- 
use data files also include the international restricted variables listed in exhibit 5-1 of 
chapter 5. 


a The TIMSS and TIMSS Advanced U.S. national restricted-use data files that are 
available through the National Center for Education Statistics. Access to these 
microdata files may be obtained by completing a restricted-use license agreement with 
NCES. The restricted-use data files are provided only on CD-ROM or DVD-ROM. 
These data files contain supplemental link files that link TIMSS or TIMSS Advanced 
school ID numbers to the school ID numbers as they appear in the publicly available 
Common Core of Data (CCD) or the Private School Universe Survey (PSS). In 
addition, race/ethnicity is provided with all available categories and free or reduced- 
price lunch is provided as a continuous variable. Because these data can reveal the 
identities of participating schools, the restricted-use data files are only made available 
to those who obtain a NCES restricted-use data license. Directions on how to obtain 
the license can be found at http://nces.ed.gov/pubsearch/licenses.asp. 


In addition to the TIMSS Advanced 2015 data files, a set of the TIMSS Advanced 1995 school and 
student data files are provided in public-use and restricted-use data formats for trend analysis purposes. 
Additional information about the TIMSS Advanced 1995 U.S. public- and restricted files is described in 
chapter 5. 


Researchers do not need to process the TIMSS microdata (public-use or restricted-use) in order to access 
and analyze the data. The TIMSS data, including Florida and national variables (race/ethnicity, free 
lunch) can be analyzed using the NCES International Data Explorer (IDE). The IDE provides for U.S. 
trend analysis as well as comparisons with all participating TIMSS countries. The IDE generates user- 
defined reports that include options for tables, gap analysis, regression analysis, and significance testing. 
The IDE can be accessed at http://nces.ed.gov/surveys/international/ide/. 


Data for Florida are available via Florida-specific, restricted-use data files. Access to these files may be 
obtained by completing a restricted-use license agreement with NCES. These restricted-use data files are 
provided only on CD-ROM or DVD-ROM. The Florida-specific, restricted-use data files contain all of 
the variables in the public released national data files plus the supplemental link files that link TIMSS 
school ID numbers to the school ID numbers as they appear in the publicly available Common Core of 
Data (CCD). Because these data can reveal the identities of participating schools, the restricted-use data 
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files are only made available to those who obtain a NCES restricted-use data license. Directions on how 
to obtain the license can be found at http://nces.ed.gov/pubsearch/licenses.asp. 


The most comprehensive explanation of the TIMSS and TIMSS Advanced international data, and hence 
of the U.S. international data file, is provided in the various TIMSS and TIMSS Advanced 2015 
publications produced by the International Association for the Evaluation of Educational Achievement 
(IEA), particularly the TIMSS 2015 User Guide for the International Database (Foy 2017a) and the 
TIMSS 2015 Advanced User Guide for the International Database (Foy 2017b). These publications 
provide detailed descriptions of the content and format of the data in the TIMSS and TIMSS Advanced 
2015 international data files and should be seen as the primary reference. The U.S. TIMSS 2015 and 
TIMSS Advanced 1995 & 2015 Technical Report and User’s Guide draws heavily on the international 
user’s guide for much of its data file-related content. This content is supplemented with detail on those 
aspects of the TIMSS and TIMSS Advanced data that were unique to the United States. 
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2. SAMPLING SCHOOLS 


21 2015 TIMSS Sample Design 


The 2015 TIMSS sample design consisted of the following samples: 
r U.S. TIMSS fourth- and eighth-grade national samples; 
rT U.S. TIMSS Advanced national sample; and 
5 TIMSS fourth- and eighth-grade Florida samples. 


The samples are described in sections 2.1.1—2.1.3. 


2.1.1 U.S. TIMSS Fourth- and Eighth-Grade National Samples 


The sample design for the fourth- and eighth-grade school samples was developed to retain most of the 
properties of the previous TIMSS U.S. school samples, and to follow international requirements as given 
in Chapter 3 of Methods and Procedures in TIMSS 2015 (Martin, Mullis, and Hooper 2016a). Like the 
2011 TIMSS sample, the U.S. sample followed a two-stage sampling process with the first stage a sample 
of schools, and the second stage a sample of classrooms within schools. All students in sampled 
classrooms were selected for assessment. 


There was no oversampling for either grade in 2015. The overlap with the 2015 National Assessment of 
Educational Progress (NAEP) school samples was minimized for the 2015 TIMSS. The TIMSS samples 
were selected before the NAEP samples due to TIMSS scheduling constraints. Thus the overlap between 
the samples was minimized when the 2015 NAEP samples were selected. 


The student population for the fourth grade 2015 TIMSS is the set of all fourth-graders in the United 
States in both public and private schools. The TIMSS school sample consists of 300 schools containing a 
fourth-grade class. The schools were selected with probability proportionate to the school’s estimated 
grade enrollment of fourth-graders from the 2015 NAEP school frame. A total of two fourth-grade 
mathematics classes were selected within each school in an equal probability sample (unless there were 
only one or two classes, in which case all fourth-grade classes were taken with certainty). The overall 
sample design was intended to approximate a self-weighting sample of students as much as possible in 
accordance with the international guidelines, with each fourth-grade student in the United States having 
an equal probability of being selected. 


The student population for the eighth-grade 2015 TIMSS is the set of all eighth-graders in the United 
States in both public and private schools. The TIMSS school sample consists of 300 schools containing an 
eighth-grade class. The schools were selected with probability proportionate to the school’s estimated 
grade enrollment of eighth-graders from the 2015 NAEP school frame. A total of two eighth-grade 
mathematics classes were selected within each school in an equal probability sample (unless there were 
only one or two classes, in which all eighth-grade classes were taken with certainty). The overall sample 
design was intended to approximate a self-weighting sample of students as much as possible in 
accordance with the international guidelines, with each eighth-grade student in the U.S. having an equal 
probability of being selected. 
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2.1.2 U.S. TIMSS Advanced National Sample 


The International Association for the Evaluation of Educational Achievement (IEA) overall design for the 
2015 TIMSS Advanced replicates the TIMSS 2008 Advanced study, in which the United States did not 
participate. However, the U.S. sample followed a two-stage sampling process with the first stage a sample 
of schools, and the second stage a sample of advanced students within schools rather than classrooms. 
The overlap with the 2015 NAEP grade 12 school sample was minimized for the 2015 TIMSS Advanced. 
The NAEP grade 12 sample was selected before the TIMSS Advanced sample. Thus the overlap between 
the samples was minimized when the TIMSS sample was selected. 


A single TIMSS school sample consisting of 348 schools containing at least one twelfth-grade class was 
selected for both subjects (advanced mathematics and physics). The schools were selected with 
probability proportionate to the school’s estimated grade enrollment of twelfth-graders from the 2015 
NAEP school frame. Within each sampled school, separate student samples were selected for advanced 
mathematics and physics in systematic equal probability samples. The student sampling algorithm was 
designed to meet international guidelines with a target of 4,000 in advanced mathematics and physics for 
a total of 8,000 sampled advanced students. 


2.1.3 TIMSS Florida Samples 


A TIMSS public school state sample was drawn for Florida at both fourth and eighth grade. The 2015 
TIMSS state sample replicated the national sample design whenever possible. The school frame to draw 
the Florida state samples was identical to the national frame of public schools in Florida. The overlap with 
the TIMSS national and 2015 NAEP school samples was minimized for the 2015 Florida samples at both 
fourth and eighth grade. The Florida state school sample consists of 54 schools at both grades four and 
eight containing a fourth- and eighth-grade class, respectively. 


The rest of this chapter describes the school samples. Section 2.2 describes the sampling frame—the data 
sources, data preparation, and the stratification variables. Section 2.3 describes the stratification. Section 
2.4 describes the national school sample selection—the measure of size, substitute schools, and selecting 
classrooms and students. Section 2.5 provides tabulations within subgroups for the frame and sample. 
Section 2.6 describes the state school sample selection. The data collection methods are covered in a 
chapter 4. 


2.2 School Sampling Frames 


The U.S. school sampling frames for fourth, eighth, and twelfth grades were developed from two national 
databases in the National Center for Education Statistics—public schools in the Common Core of Data 
(CCD—http://nces.ed.gov/ccd/) and private schools in the Private School Survey (PSS— 
http://nces.ed.gov/surveys/pss/). These sources provide full coverage of all fourth-, eighth- and twelfth- 
grade students in the education system in the United States. The TIMSS school frames were constructed 
using the 2012-13 CCD and the 2011-12 PSS, the most current data at the time of the frame construction. 


The data preparation of the school frames benefited from procedures developed for the National 
Assessment of Educational Progress (NAEP), a large educational survey in the United States. The school 
frames used the NAEP 2015 school frame as an input data source. 
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Eligible schools in the school frames included schools operating in the 50 states and the District of 
Columbia, Department of Defense (DoD) domestic schools, and Bureau of Indian Education (BIE) 
schools. Schools in Puerto Rico and U.S. territories, DoD schools overseas, adult education institutions 
with no fourth-, eighth- or twelfth-grade students, and non-education institutions (e.g., home schools and 
correspondence schools) were ineligible for the study. 


2.2.1 Fourth-Grade Frame 


Any school having a fourth grade was included on the TIMSS fourth-grade school sampling frame. Table 
2-1 presents frame tabulations of the number of schools by the school grade span (lowest to highest grade 
level of the school). 


Table 2-1. Number and percentage of students and schools included in the U.S. TIMSS fourth-grade 
school sampling frame, by grade span: 2015 


Grade span Students Percent Schools Percent 

Total 4,079,075 100 70,852 100 
Grades 1 to 5 2,038,439 50.0 26,321 37.1 
Grades 1 to 6 651,996 16.0 11,558 16.3 
Grades 1 to 8 477,099 11.7 15,194 21.4 
Grades 1 to 12 142,453 3.5 6,206 8.8 
Other 769,088 18.9 11,573 16.3 


NOTE: Detail may not sum to totals because of rounding. The “other” grade spans includes all additional grade spans. 
SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS), 2015. 


2.2.2 Eighth-Grade Frame 


Any school having an eighth grade was included on the TIMSS eighth-grade school sampling frame. 
Table 2-2 presents frame tabulations of the number of schools by the school grade span (lowest to highest 
grade level of the school). 


Table 2-2. Number and percentage of students and schools included in the U.S. TIMSS eighth-grade 
school sampling frame, by grade span: 2015 


Grade span Students Percent Schools Percent 

Total 4,020,280 100 46,207 100 
Grades 1 to 8 477,099 11.9 15,194 32.9 
Grades 6 to 8 2,141,606 53.3 10,132 21.9 
Grades 1 to 12 142,453 3.5 6,206 13.4 
Grades 7 to 12 139,056 3.5 3,128 6.8 
Grades 7 to 8 665,849 16.6 2,694 5.8 
Other 454,217 11.3 8,853 19.2 


NOTE: Detail may not sum to totals because of rounding. The “other” grade spans includes all additional grade spans. 
SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS), 2015. 
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2.2.3 Advanced Frame 


Any school having a twelfth grade was included on the TIMSS Advanced school sampling frame. Table 
2-3 presents frame tabulations of the number of schools by the school grade span (lowest to highest grade 
level of the school). 


Table 2-3._Number and percentage of students and schools included in the U.S. TIMSS Advanced 
school sampling frame, by grade span: 2015 


Grade span Students Percent Schools Percent 

Total 3,806,305 100 33,411 100 
Grades 1 to 12 160,923 4.2 6,206 18.6 
Grades 6 to 12 66,667 1.8 1,540 4.6 
Grades 7 to 12 153,439 4.0 3,128 9.4 
Grades 9 to 12 3,090,729 81.2 16,274 48.7 
Grades 10 to 12 161,259 4.2 997 3.0 


Other 173,288 4.6 9,266 15.8 


NOTE: Detail may not sum to totals because of rounding. The “other” grade spans includes all additional grade spans. 
SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS), 2015. 


To supplement the twelfth-grade frame data, NCES worked with the College Board to provide a file with 
data on schools that offered advanced placement (AP) courses in 2013. The College Board data 
supplemented the frame data with school information on students who took AP exams. The College 
Board file included school information such as the NCES School ID and school name and address that 
was used in matching to the school frame. The data on the College Board file included frequencies of 
students taking AP exams in calculus, physics, and both calculus and physics. 


The 2013 College Board file was merged to the twelfth-grade frame first by NCES ID and then by school 
name and address when possible. Approximately 97 percent of over 13,100 schools listed on the College 
Board file were matched to the frame. The matched schools constituted almost 40 percent of the over 
33,000 schools on the frame. The schools that matched to the College Board file will hereinafter be called 
“AP schools” and the nonmatches “non-AP schools.” 


Counts of eligible advanced mathematics and physics students were needed to estimate the number of 
advanced students that would be sampled to ensure the U.S. design met the international target sample 
sizes of advanced mathematics and physics students. However, only grade enrollment data were available 
by school on the school frame, not information on courses that students were students enrolled in. 
Because these actual counts of advanced mathematics and physics students were not available on the 
school frame, estimates of eligible student counts were computed with the available information. Besides 
the AP frequencies from the College Board, the only information available was the percentage of 
graduates who earned credit in calculus and/or physics according to the 2009 High School Transcript 
Study (HSTS). Table 2-4 shows the 2009 HSTS percentages of graduates who earned credit in calculus 
and/or physics by the grade in which the course was completed. For example, it shows that 16.8 percent 
of all graduates completed a course in calculus with 15.2 percent of twelfth-graders completing the course 
in twelfth grade, 2.8 percent of eleventh-graders completing the course in eleventh grade, and 0.1 percent 
of ninth/tenth-graders completing the course in ninth/tenth grade. 


? The percentages are not additive to the overall percentage of graduates because the denominator for each grade level is different. 
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In non-AP schools, estimates of students in calculus and/or physics were calculated by multiplying the 
grade 12 enrollment by the total percentage of graduates who earned credit in calculus and/or physics 
according to the HSTS shown in table 2-4. For example, if the grade 12 enrollment in a school was 100, 
then an estimated 16.8 and 4.6 percent of students earned credit in calculus and physics, respectively. In 
AP schools, the frequencies of students taking AP exams in calculus, physics, and both calculus and 
physics were adjusted based on the HSTS percentages. This was done within each school by adjusting the 
AP counts in calculus, physics, and both calculus and physics by the comparable ratio of total percentage 
of advanced to AP students. For example, the number of advanced mathematics students in a school was 
estimated by multiplying the AP calculus count in the school (from the College Board) by a ratio of 
16.8/11.3 (i.e., the ratio of the total percentage of calculus graduates to the percentage of AP calculus 
graduates from table 2.4). 


Table 2-4. HSTS 2009: Percentages of graduates who earned credit in calculus and/or physics by grade 
the course was completed: 2009 


Course Overall Twelfth Eleventh Ninth/Tenth 
Calculus 16.8 15.2 2.8 0.1 
AP calculus 11.3 10.0 2.0 0.1 
Advanced physics 4.6 3.4 1.3 0.1 
AP physics 3.6 2.7 1.0 0.0 

Calculus and advanced physics 
(overlap) 3.4 T T T 

AP calculus and AP physics 
(overlap) 2.6 T T T 


+ Not applicable. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, High School Transcript Study (HSTS), 2009. 


The estimated average frame counts of eligible students using the algorithm described above are shown in 
table 2-5 by enrollment and College Board file match status. Even though less than 40 percent of the 
schools on the frame were AP schools, the vast majority of the grade 12 and advanced students were from 
the AP schools as the AP schools were on average much larger. For example, the average number of 
advanced mathematics students in all schools was 19.9, but in AP schools it was 39.5. 


Table 2-5. Average number of students per school by enrollment and College Board file match status 


All 
Enrollment schools | Non-AP schools AP schools 
Grade 12 116.3 43.7 230.1 
Advanced mathematics total 19.9 7.4 39.5 
Advanced mathematics only 16.7 5.9 33.6 
Advanced physics total 5.5 2.0 11.0 
Advanced physics only 2.3 0.5 5.1 
Advanced mathematics and physics 3.2 1.5 5.9 


SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS), 2015. 
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2.3 Stratification 


The sample design was a stratified systematic sample within each stratum, with sampling probabilities 
proportional to size (PPS). Stratification was used for sample efficiency and consistency with previous 
designs. For grades 4 and 8, twelve explicit strata were formed by the following variables, shown in 
alphabetical order: 


7 census region—Northeast, Midwest, South, and West; 


a poverty level’—for public schools, “high” poverty is defined as having 50 percent or 
more of the students eligible for participation in the free or reduced-price lunch 
program (FRPL),‘ and “low” poverty is defined as having less than 50 percent 
eligible; and 


a school type—school is either under public control (operated by publicly elected or 
appointed officials) or private control (operated by privately elected or appointed 
officials and derives its major source of funds from private sources). 


For the Advanced sample, nine explicit strata? were formed by the following variables shown in 
alphabetical order: 


a AP status—indicates whether or not the school had students who took a calculus, 
physics, or both calculus and physics AP test in 2013; 


7 census region—Northeast, Midwest, South and West; and 


a School type—school is either under public control (operated by publicly elected or 
appointed officials) or private control (operated by privately elected or appointed 
officials and derives its major source of funds from private sources). 


Within each stratum for all three samples, the frame was implicitly stratified by the following three 
categorical stratification variables: 


7 locale—urban-centric locale code (i.e., city, suburb, town, rural); 
: race/ethnicity status—student population in the school is “15 percent or above” or 
“below 15 percent” Black, Hispanic, Asian, Hawaiian/Pacific Islander, American 


Indian and Alaska Native students, and multiracial students;°® and 


7 estimated grade enrollment. 


3 The sample frame did not contain a direct measure of poverty. No free or reduced-price lunch program (FRPL) data were available for private 
schools; thus all private schools are treated as low-poverty schools. 

4 FRPL refers to the same program that is sometimes called the National School Lunch Program (NSLP). TIMSS has traditionally used the name 
FRPL and that will be used in this report hereinafter. 

5 Some variables were collapsed when forming the explicit strata due to small samples sizes per international requirements, so a full cross- 
classification of 16 strata was not possible. 

6 Black includes African American and Hispanic includes Latino. Racial categories exclude Hispanic origin. 
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21 Fourth-Grade Stratification 


The following tables show the total number and percentage of fourth-grade students and schools in the 
TIMSS 2015 school frame by census region (table 2-6), poverty level (table 2-7), school type (table 2-8), 
locale (table 2-9), race/ethnicity status (table 2-10), and by poverty level, school type, and region (table 
2-11). 


Table 2-6. Number and percentage of students and schools included in the U.S. TIMSS fourth-grade 
school sampling frame, by region: 2015 


Region Students Percent Schools Percent 

Total 4,079,075 100 70,852 100 
Northeast 659,672 16.2 12,295 17.4 
Midwest 866,815 21.3 17,579 24.8 
South 1,570,084 38.5 23,928 33.8 
West 982,504 24.1 17,050 24.1 


NOTE: Detail may not sum to totals because of rounding. Region of country is based on Census Bureau definitions. 
SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS), 2015. 


Table 2-7. Number and percentage of students and schools included in the U.S. TIMSS fourth-grade 
school sampling frame, by poverty level: 2015 


Poverty level Students Percent Schools Percent 

Total 4,079,075 100 70,852 100 
High 2,021,124 49.5 28,299 39.9 
Low 2,057,951 50.5 42,553 60.1 


NOTE: For public schools, “high” poverty is defined as having 50 percent or more of the students eligible for participation in the free or reduced- 
price lunch program (FRPL), and “low” poverty is defined as having less than 50 percent eligible. Because no FRPL data were available for 
private schools, all private schools are categorized as “low.” 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS), 2015. 


Table 2-8. Number and percentage of students and schools included in the U.S. TIMSS fourth-grade 
school sampling frame, by school type: 2015 


School type Students Percent Schools Percent 

Total 4,079,075 100 70,852 100 
Private 341,848 8.4 19,674 27.8 
Public Salgled 91.6 51,178 72.2 


SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS), 2015. 


N 
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Table 2-9. Number and percentage of students and schools included in the U.S. TIMSS fourth-grade 
school sampling frame, by locale: 2015 


Locale Students Percent Schools Percent 

Total 4,079,075 100 70,852 100 
City 1,285,339 31.5 21749 30.7 
Suburb 1,625,208 39.8 23391 33.0 
Town 444,688 10.9 7561 10.7 
Rural 723,840 17.7 18151 25.6 


NOTE: Detail may not sum to totals because of rounding. For definitions of these urban-centric locales, see 
http://nces.ed.gov/surveys/urbaned/definitions.asp. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS), 2015. 


Table 2-10. Number and percentage of students and schools included in the U.S. TIMSS fourth-grade 
school sampling frame, by race/ethnicity status: 2015 


Race/ethnicity status Students Percent Schools Percent 

Total 4,079,075 100 70,852 100 
15 percent or above 3,198,363 78.4 49,838 70.3 
Below 15 percent 880,712 21.6 21,014 29.7 


NOTE: Race/ethnicity status refers to the percentage of Black, Hispanic, Asian, Hawaiian/Pacific Islander, American Indian and Alaska Native 
students, and multiracial students. Black includes African American and Hispanic includes Latino. Racial categories exclude Hispanic origin. 
SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS), 2015. 


Table 2-11. Number and percentage of students and schools included in the U.S. TIMSS fourth-grade 
school sampling frame, by region, poverty level, and school type: 2015 


Region Poverty level School type Students Percent Schools Percent 

Total 4,079,075 100 70,852 100 
Northeast High Public 228,005 5.6 3,078 4.3 
Midwest High Public 353,088 8.7 6,061 8.6 
South High Public 930,295 22.8 12,002 16.9 
West High Public 509,736 12.5 7,158 10.1 
Northeast Low Private 75,876 1.9 4,377 6.2 
Midwest Low Private 86,309 2.1 5,026 7.1 
South Low Private 114,175 2.8 6,375 9.0 
West Low Private 65,488 1.6 3,896 5.9 
Northeast Low Public 355,791 8.7 4,840 6.8 
Midwest Low Public 427,418 10.5 6,492 9.2 
South Low Public 525,614 12.9 5,551 7.8 
West Low Public 407,280 10.0 5,996 8.5 


a i) 
NOTE: Region of country is based on Census Bureau definitions. For public schools, “high” poverty is defined as having 50 percent or more of 
the students eligible for participation in the free or reduced-price lunch program (FRPL), and “low” poverty is defined as having less than 

50 percent eligible. Because no FRPL data were available for private schools, all private schools are categorized as “low.” 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS), 2015. 
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3.2 Eighth-Grade Stratification 


The following tables show the total number and percentage of eighth grade students and schools in the 
TIMSS 2015 eighth-grade school frame by census region (table 2-12), poverty level (table 2-13), school 
type (table 2-14), locale (table 2-15), race/ethnicity status (table 2-16), and by poverty level, school type, 
and region (table 2-17). 


Table 2-12. Number and percentage of students and schools included in the U.S. TIMSS eighth-grade 
school sampling frame, by region: 2015 


Region Students Percent Schools Percent 

Total 4,020,280 100 46,207 100 
Northeast 665,312 16.5 8,629 18.7 
Midwest 868,161 21.6 12,271 26.6 
South 1,533,061 38.1 14,880 32.2 
West 953,746 23.7 10,427 22.6 


NOTE: Detail may not sum to totals because of rounding. Region of country is based on Census Bureau definitions. 
SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS), 2015. 


Table 2-13. Number and percentage of students and schools included in the U.S. TIMSS eighth-grade 
school sampling frame, by poverty level: 2015 


Poverty level Students Percent Schools Percent 

Total 4,020,280 100 46,207 100 
High 1,817,810 45.2 15,079 32.6 
Low 2,202,470 54.8 31,128 67.4 


NOTE: For public schools, “high” poverty is defined as having 50 percent or more of the students eligible for participation in the free or reduced- 
price lunch program (FRPL), and “low” poverty is defined as having less than 50 percent eligible. Because no FRPL data were available for 
private schools, all private schools are categorized as “low.” 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS), 2015. 


Table 2-14. Number and percentage of students and schools included in the U.S. TIMSS eighth-grade 
school sampling frame, by school type: 2015 


School type Students Percent Schools Percent 

Total 4,020,280 100 46,207 100 
Private 331,909 8.3 17,891 38.7 
Public 3,688,371 91.7 28,316 61.3 


SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS), 2015. 
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Table 2-15. Number and percentage of students and schools included in the U.S. TIMSS eighth-grade 
school sampling frame, by locale: 2015 


Locale Students Percent Schools Percent 

Total 4,020,280 100 46,207 100 
City 1,218,578 30.3 13,367 28.9 
Suburb 1,606,624 40.0 13,283 28.7 
Town 457,504 11.4 5,316 11.5 
Rural 737,574 18.3 14,241 30.8 


NOTE: Detail may not sum to totals because of rounding. For definitions of these urban-centric locales, see 
http://nces.ed.gov/surveys/urbaned/definitions.asp. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS), 2015. 


Table 2-16. Number and percentage of students and schools included in the U.S. TIMSS eighth-grade 
school sampling frame, by race/ethnicity status: 2015 


Race/ethnicity status Students Percent Schools Percent 

Total 4,020,280 100 46,207 100 
15 percent or above 3,112,845 77.4 30,274 65.5 
Below 15 percent 907,435 22.6 15,933 34.5 


NOTE: Race/ethnicity status refers to the percentage of Black, Hispanic, Asian, Hawaiian/Pacific Islander, American Indian and Alaska Native 
students, and multiracial students. Black includes African American and Hispanic includes Latino. Racial categories exclude Hispanic origin. 
SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS), 2015. 


Table 2-17. Number and percentage of students and schools included in the U.S. TIMSS eighth-grade 
school sampling frame, by region, poverty level, and school type: 2015 


Region Poverty level School type Students Percent Schools Percent 

Total 4,020,280 100 46,207 100 
Northeast High Public 205,520 5.1 1,852 4.0 
Midwest High Public 303,456 7.5 3,537 7.7 
South High Public 847,165 21.1 6,233 13.5 
West High Public 461,669 11.5 3,457 7.5 
Northeast Low Private 76,345 1.9 4,219 9.1 
Midwest Low Private 81,645 2.0 4,410 9.5 
South Low Private 110,085 2.7 5,710 12.4 
West Low Private 63,834 1.6 3,592 BA 
Northeast Low Public 383,447 9.5 2,998 5.5 
Midwest Low Public 483,060 12.0 4,324 9.4 
South Low Public 575,811 14.3 2,937 6.4 
West Low Public 428,243 10.7 3,418 7.4 


Ln i i 
NOTE: Detail may not sum to totals because of rounding. Region of country is based on Census Bureau definitions. For public schools, “high” 
poverty is defined as having 50 percent or more of the students eligible for participation in the free or reduced-price lunch program (FRPL), and 
“low” poverty is defined as having less than 50 percent eligible. Because no FRPL data were available for private schools, all private schools are 
categorized as “low.” 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS), 2015. 
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2.3.3 Advanced Stratification 


The following tables show the total number and percentage of twelfth-grade students and schools in the 
TIMSS 2015 Advanced school frame by AP status level (table 2-18), census region (table 2-19), school 
type (table 2-20), locale (table 2-21), race/ethnicity status (table 2-22), and by AP status, region, and 


school type (table 2-23). 


Table 2-18. Number and percentage of students and schools included in the U.S. TIMSS Advanced 


school sampling frame, by AP status: 2015 


AP status Students Percent 

Total 3,806,305 100 
AP school 2,942,935 77.3 
Non-AP school 863,370 2217 


Schools Percent 
33,411 100 
12,789 38.3 
20,622 61.7 


NOTE: AP status indicates whether or not the school had students who took a calculus, physics, or both calculus and physics AP test in 2013. 
SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 


Study (TIMSS), 2015. 


Table 2-19. Number and percentage of students and schools included in the U.S. TIMSS Advanced 


school sampling frame, by region: 2015 


Region Students Percent 

Total 3,806,305 100 
Northeast 667,150 17.5 
Midwest 836,147 22.0 
South 1,330,111 34.9 
West 972,897 25.6 


Schools Percent 
33,411 100 
5,599 16.8 
8,435 25.2 
11,483 34.4 
7,894 23.6 


NOTE: Region of country is based on Census Bureau definitions. 


SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 


Study (TIMSS), 2015. 


Table 2-20. Number and percentage of students and schools included in the U.S. TIMSS Advanced 


school sampling frame, by school type: 2015 


School type Students Percent 

Total 3,806,305 100 
Private 314,870 8.3 
Public 3,491,435 91.7 


Schools Percent 
33,411 100 
9,776 29.3 
23,635 70.7 


SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 


Study (TIMSS), 2015. 
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Table 2-21. Number and percentage of students and schools included in the U.S. TIMSS Advanced 
school sampling frame, by locale: 2015 


Locale Students Percent Schools Percent 


Total 3,806,305 100 33,411 100 
City 1,174,938 30.9 8967 26.8 
Suburb 1,526,987 40.1 9109 27.3 
Town 418,424 11.0 4487 13.4 
Rural 685,956 18.0 10848 32.5 


NOTE: For definitions of these urban-centric locales, see http://nces.ed.gow/surveys/urbaned/definitions.asp. 
SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS), 2015. 


Table 2-22. Number and percentage of students and schools included in the U.S. TIMSS Advanced 
school sampling frame, by race/ethnicity status: 2015 


Race/ethnicity status Students Percent Schools Percent 

Total 3,806,305 100 33,411 100 
15 percent or above 2,924,205 76.8 22,419 67.1 
Below 15 percent 882,100 23.2 10,992 32.9 


NOTE: Race/ethnicity status refers to the percentage of Black, Hispanic, Asian, Hawaiian/Pacific Islander, American Indian and Alaska Native 
students, and multiracial students. Black includes African American and Hispanic includes Latino. Racial categories exclude Hispanic origin. 
SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS), 2015. 


Table 2-23. Number and percentage of students and schools included in the U.S. TIMSS Advanced 
school sampling frame, by AP status, region, and school type: 2015 


School 

AP Status Region type Students Percent Schools Percent 

Total 3,806,305 100 33,411 100 
AP School Northeast Public 478,899 12.6 2,073 6.2 
AP School Midwest Public 552,142 14.5 2,393 7.2 
AP School South Public 990,211 26.0 3,637 10.9 
AP School West Public 704,829 18.5 2,177 6.5 
AP School Northeast Private 59,380 1.6 618 1.8 
AP School Midwest Private 48,074 1.3 489 1.5 
AP School South Private 65,269 1.7 894 2.7 
AP School West ‘Private 44,131 1.2 508 1.5 
Non-AP School Northeast Public 101,735 2.7 1,373 4.1 
Non-AP School Midwest Public 216,661 5.7 4,306 12.9 
Non-AP School South — Public 237,494 6.2 3,936 11.8 
Non-AP School West Public 209,464 5.5 3,740 11.2 


See notes at end of table. 


U.S. TIMSS 2015 and TIMSS Advanced 1995 & 2015 Technical Report and User’s Guide 2-12 


Table 2-23. Number and percentage of students and schools included in the U.S. TIMSS Advanced 
school sampling frame, by AP status, region, and school type: 2015—Continued 


School 
AP Status Region type Students Percent Schools Percent 
Non-AP School Northeast Private 27,136 0.7 1,535 4.6 
Non-AP School Midwest Private 19,270 0.5 1,247 3.7 
Non-AP School South Private 37,137 1.0 3,016 9.0 
Non-AP School West Private 14,473 0.4 1,469 4.4 


NOTE: Detail may not sum to totals because of rounding. Region of country is based on Census Bureau definitions. AP status refers whether or 
not the school had students who took a calculus, physics, or both calculus and physics AP test in 2013. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS), 2015. 


2.4 National School Samples 


2.4.1 Measure of Size 


The goal for the TIMSS sample was to attain a self-weighting student sample. To achieve this, schools’ 
probability of selection was related to their measure of size (MOS), which is proportional to its share of 
the target population, that is, the fourth-, eighth-, or twelfth-grade enrollments. This method also reduces 
the chance of selection for smaller schools. This improves cost efficiency by increasing the number of 
students per school. However, students in schools with enrollments of only a few students would have 
very large weights if selected. To minimize the impact of these small schools on variances and estimates, 
the minimum measure of size was set to 5. 


The following is a summary of the steps for assigning measures of size to the schools on the TIMSS 
frames. 


Determine the estimated target population size for the school. This is the estimated enrollment per grade 
(4, 8, or 12) from the school frame. If the grade enrollment was not available, it was calculated by 
dividing the school’s total enrollment by the number of grades in the school. 


The measure of size according to the estimated enrollment per grade is 5 if the grade enrollment is less 
than or equal to 5, and is equal to the grade enrollment (grade 4, grade 8, or grade 12) otherwise. 


2.4.2 Fourth- and Eighth-Grade Samples 


The U.S. national fourth- and eighth samples used a two-stage design—a stratified systematic sample of 
schools with sampling probabilities proportional to size (PPS) and then classes within sampled schools. 
The school selection probability was configured such that all fourth- or eighth-grade students in the 
United States would have approximately equal probability of being selected in the samples. Note that in 
large schools, a smaller proportion of the classes (and therefore of the students) is selected, but this 
lower rate of selecting students in large schools is offset by a larger probability of selection of large 
schools, as schools are selected with probability proportional to size. A sample of 300 schools was 
drawn from the fourth- and eighth grade frames. A systematic sample was selected independently in each 
stratum where the measure of size is the estimated number of students in fourth or eighth grade. 
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Within each explicit stratum, the frames were implicitly stratified by two categorical stratification 
variables. The order of the stratification is not given because of confidentiality concerns. Each frame was 
sorted in alternating (serpentine) sort order according to these school characteristics, implicitly stratifying 
the frame. The last sort within the implicit stratification was by grade enrollment (measure of size or 
MOS) in descending order. Alternating the “sort order” sorts a frame from lowest to highest value with 
respect to the first sort variable, then within each level of the first sort variable, the second sort variable 
alternates its sort order, from lowest to highest for the first level of the first sort variable, then from 
highest to lowest for the second level of the first sort variable, then again from lowest to highest for the 
third level of the first sort variable, and so on. Each of the variables will alternate the sort order within 
each level of the preceding sort variable. This means that schools adjacent on the list are not substantially 
different or at most different by one sorting characteristic. 


2.4.3 Advanced Sample 


The target population for advanced mathematics was “students in the final year of secondary schooling 
who have taken courses in advanced mathematics.” The target population for physics was “students in the 
final year of secondary schooling who have taken courses in physics.” The courses that define the target 
populations have to cover most, if not all, of the advanced mathematics and physics topics that were 
outlined in the TIMSS Advanced 2015 Assessment Frameworks. 


The IEA TIMSS Advanced framework specifies, at the highest level, that the target population includes 
students who have taken courses that contain the following content, respectively: 


7 advanced mathematics content: 
- algebra; 
- geometry; and 
- calculus; 
5 physics content: 
- mechanics and thermodynamics; 
- electricity and magnetism; and 
- wave phenomena and atomic/nuclear physics. 


In some countries the target populations are 100 percent overlapping, and in others, like the United States, 
they are overlapping to a variable degree. 


Defining the target population in the United States is more challenging than in most countries. The United 
States does not have a common curriculum across the country and students do not follow a uniform 
course-taking sequence. Therefore, students may have taken these courses prior to their senior year, and 
many different courses may meet the framework criteria. Complicating things further, the school lists that 
form the sampling frames are obtained from the NCES CCD (public school) and PSS (private school) 
files, neither of which includes courses offered in the schools. Also, students may take these courses in 
their high schools, or in colleges/universities, or virtually. For the purpose of this study, eligibility for 


U.S. TIMSS 2015 and TIMSS Advanced 1995 & 2015 Technical Report and User’s Guide 2-14 


advanced mathematics was defined as have taken or were taking a calculus course, and for physics as 
have taken or were taking an advanced physics course similar to AP physics. 


In order to quantify the proportion of the school-leaving age cohort taking advanced mathematics and 
physics courses, TIMSS Advanced computed a TIMSS Advanced Mathematics Coverage Index 
(TAMC]I) and a TIMSS Advanced Physics Coverage Index (TAPCI) for each participating country. In 
part, these indices reflect the overall sampling coverage of the respective populations in each country; but, 
more importantly, they show that only a very select group of final-year students were considered eligible 
for TIMSS Advanced 2015, and that the percentages of these students varied across countries. 


The TIMSS Advanced coverage indices for advanced mathematics and physics were defined as follows: 


Estimated total number of students in the advanced mathematics population 
TAMC] =| __W_T___W__W1___Y______1___1+__1__1___1_—_+_——. x 100% 
Total national population in the corresponding age cohort 


Estimated total number of students in the advanced physics population 
TAP CL = ATT ________________—. x 100% 
Total national population in the corresponding age cohort 


The numerator in each index is the total number of students eligible for TIMSS Advanced 2015, either for 
advanced mathematics or physics, as estimated from the weighted samples. The denominator is an 
estimate of the size of the eligible age cohort in 2015 corresponding to the mean age of the target 
population. The final-year age cohort for each country was defined to be the age corresponding to its 
average age at the time of testing, as estimated from the weighted samples, and its size was estimated 
from national census figures. The estimated target populations were estimated from the weighted samples. 
The TIMSS Advanced coverage indices for advanced mathematics and physics for the U.S. were 11.4 and 
4.8 percent, respectively. 


The overlap with the 2015 NAEP school sample was minimized for the 2015 TIMSS Advanced. The 
NAEP grade 12 sample was selected before the TIMSS Advanced sample, and therefore the overlap 
between the samples was minimized when the TIMSS sample was selected. The Advanced sample was 
selected using a version of the Keyfitz procedure (Keyfitz 1951); Chowdhury, Chu, and Kaufman (2000) 
have described the implementation of the procedure. The method is used to minimize overlap between 
one or more surveys. By minimizing the overlap with the NAEP sample, the assessed students could be 
included in only one study with proper probabilities. This was accomplished by partitioning the frame 
into the following two groups shown in order as in table 2 of the paper. The two groups were as follows: 


1. Schools not selected for the NAEP sample; and 
2. Schools selected for the NAEP sample. 


With this design, the method accomplished the goal of selecting the entire state samples from group 1 and 
none from group 2. For the schools in group 2, this was due to the sum of the school’s probabilities of 
being selected for the state sample and the national sample was always less than one. In that case, their 
conditional probabilities are zero. 


Unlike the fourth and eighth grade designs, there was oversampling for the Advanced sample in 2015 by 
oversampling AP schools in order to increase the overall yield of advanced students. The oversampling 
rate was set so that non-AP schools were undersampled at half the rate of the AP schools that effectively 
oversampled the AP schools. Oversampling the AP schools increased the student sample yield by 
sampling more schools with a larger enrollment of eligible students, and undersampling the non-AP 
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schools lessened the number of sampled schools that likely contained no eligible students. The 
oversampling was set at this rate and not at a higher rate due to the increase in design effect of the 
unequal weights. Despite the fact that not all of the non-AP schools had any eligible students, they still 
needed to be given a chance of selection as it was not possible to identify those schools definitively on the 
frame. Excluding those schools would not meet the international sampling guidelines. 


The Advanced sample used a two-stage design—a stratified systematic sample of schools with sampling 
probabilities proportional to size (PPS) and then students within sampled schools. A systematic sample 
was Selected independently in each stratum with PPS sampling where the measure of size was the 
estimated number of students in twelfth grade. 


The TIMSS Advanced main study sample consisted of high schools with a target of 4,000 in advanced 
mathematics and physics for a total of 8,000 sampled students and a goal of 3,600 assessed students per 
subject. A sample of 350 schools was targeted in an effort to ensure 300 eligible and participating schools 
where school eligibility was determined by offering at least one of the eligible courses for advanced 
mathematics and physics. A sample of 348 were sampled which varied slightly from the target due to the 
overlap control. Undersampling the non-AP schools by a factor of one-half resulted in 304 and 44 AP and 
non-AP sample schools, respectively. Course catalogues were obtained from all participating schools, and 
their calculus and advanced physics courses were coded to determine eligible schools. Mathematics and 
science department chairs were also asked to identify eligible courses from their catalogues. 


2.4.4 Substitute Schools 


Although efforts were made to secure the participation of all schools selected, it was anticipated at the 
time of sampling that not all schools would choose to participate. Therefore, as each school was selected 
for a sample, the two neighboring schools in the sampling frame were designated as substitute schools. 
The first school following the sample school was the first substitute, and the first school preceding it was 
the second substitute. If an original school refused to participate, the first substitute was then contacted. If 
that school also refused to participate, the second substitute was then contacted. 


There were several constraints on the assignment of substitutes. One sampled school was not allowed to 
be a substitute for another, and a given school could not be assigned to be a substitute for more than one 
sampled school. Furthermore, substitutes were required to be in the explicit stratum as the sampled 
school. If the sampled school was the first or last school in the stratum, then the second school following 
or preceding the sampled school was identified as the substitute. If the first substitute school did not have 
the same implicit stratification values as the sampled school, the first and second substitute were 
switched. With international consent, the substitute school assignment also avoided the NAEP sample 
schools whenever possible within the above rules. Under these rules, it was possible to identify two 
substitutes for each sampled school. 


2.4.5 Selecting Classrooms—Grades 4 and 8 


The final stage of selection for grades 4 and 8 was of classrooms within schools. Within each sampled 
school that agreed to participate in TIMSS at fourth or eighth grade, all classrooms in the school were 
listed on the classroom sampling frame. Classroom lists were gathered from participating schools 
electronically using an adaptation of a secure E-Filing process. E-Filing was successfully used in the 
Program for International Student Assessment (PISA) 2012, and provides advantageous features such as 
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efficiency and data quality checks. Schools accessed the E-Filing system through the MyTIMSS.com 
website. Once the list of classrooms was received from a school, it was formatted for importing into 
WinW33S, the international sampling and data management software. 


Schools were asked to indicate the names of all classes containing fourth- or eighth-grade students, the 
number of fourth- or eighth-grade students in the class, and whether it was a “special class” in which all 
or most of the students were learning disabled or classified as having limited English proficiency. Even 
though TIMSS does provide accommodations, classrooms were excluded from the subsequent classroom 
sampling if all or most of the students were learning disabled. Classrooms with fewer than 15 students 
were collapsed into pseudoclassrooms so that each classroom on the school’s classroom sampling frame 
had at least 20 students.’ An equal probability sample of two classrooms or pseudoclassrooms was 
sampled from the classroom frame for each school. All students in sampled classrooms 
(pseudoclassrooms) were selected for assessment. 


2.4.6 Selecting Students—Advanced 


The second stage of selection for the TIMSS Advanced sample was of students within schools. Within 
each sampled school that agreed to participate in TIMSS Advanced, all eligible advanced mathematics 
and physics students in the school were listed on the student sampling frame. Similarly to the method 
described for grades 4 and 8, student lists were gathered from participating schools electronically using an 
adaptation of a secure E-Filing process. However, once the list of students was received from a school, it 
was not imported into WinW3S software, which only sampled classrooms. The student sampling was 
designed to meet international guidelines described earlier. The students were sampled within each 
school using the following algorithm: 


1. Select all students eligible for both advanced mathematics and physics, up to a total of 
40. Otherwise take a sample of size 40. 


2. Assign one-fifth of the doubly eligible students to the mathematics assessment and the 
rest to physics. 


3. Select all students eligible for physics only, up to a total of 20. Otherwise take a 
sample of size 20. 


4. Select one-quarter of the mathematics-only students, subject to a maximum of 40 and 
a minimum of 10 (or all students if there are fewer than 10). 


Based on data from 2009 HSTS (see Table 2-4), on average in a school with 500 grade 12 students the 
expected eligibility was as follows: 


A. 3.4 percent of graduates earned credit in both calculus and advanced physics. 
Therefore, 17 students are eligible for both mathematics and physics. All would be 
sampled, and 3 or 4 assigned to mathematics and 13 or 14 to physics. 


7 Pseudoclasses were automatically created during the sampling of classes if there were small classes in a school. A small class was defined as a 
class for which the number of students was smaller than half the average class size in the explicit stratum across all schools. In such a case, 
WinW3S automatically sampled two or more classes that were grouped together to form a pseudoclass. 
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B. 4.6 percent of graduates earned credit in advanced physics, but 3.4 percent earned 
credit in calculus as well. Therefore 1.2 percent (4.6—3.4) of graduates, or 6 students, 
are eligible for physics only. All would be sampled for physics. 


C. 16.8 percent of graduates earned credit in calculus, but 3.4. percent earned credit in 
physics as well. There 13.4 percent (16.8—3.4) or 67 students are eligible for 
mathematics only. Either 16 or 17 would be sampled for mathematics. 


Therefore, 39 or 40 students from such a school would be sampled, with about equal numbers assessed in 
mathematics and physics. 


The maximum sample size in any school was 100 students. This would be in a very unusual school, that 
had at least 40 double specialists, at least 20 physics-only specialists (or 40), and at least 160 
mathematics-only specialists; so there would be at least 220 specialists, when specialists are 18 percent of 
the total U.S. student population. 


2.5 Tabulations Within Subgroups for Frame and Sample 


This section provides an overview of the frame and sample for the explicit and implicit strata used in the 
sample process. The PPS sampling and stratification worked effectively: the sample percentage of schools 
is close to the measure-of-size (MOS) percentage of the frame for all the implicit strata. For these strata- 
defining subgroups, tables 2-24 through 2-41 present the following summary tabulations in these 
subgroups: 


7 Total measure of size. This is the summation of MOS; over the subgroup. Note that 
this is larger than the national population student size because the minimum MOS; is 


set to 5 for small schools; and 


7 Sample size. This is the final realized sample size of schools in the subgroup for the 
U.S. TIMSS fourth- or eighth-grade samples. 


2.5.1 Fourth-Grade Tabulations 


This section provides an overview of the fourth-grade frame and sample distribution by each of the 
stratification variables. Each table shows the total number and percentage of fourth-grade students in the 
sampling frame (data shown in tables 2-24 through 2-29) and the total number and percentage of schools 
in the TIMSS school sample.® By each stratification variable, the tables are census region (table 2-24), 
poverty level (table 2-25), school type (table 2-26), locale (table 2-27), race/ethnicity status (table 2-28), 
and by poverty level, school type, and region (table 2-29). 


8 The measure of size (MOS) is defined as the estimated number of students enrolled in the target grade with a minimum of 5 students per school. 
These are consistently larger than the estimated student sample size (reported in tables 2-1 through 2-14), which is the estimate of the number of 
students in the sampled schools and has no minimum per school. 
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Table 2-24. Number and percentage of students in the sampling frame and number and percentage of 
schools in the sample, U.S. TIMSS fourth grade, by region: 2015 


Frame Sample 
Region Measure of size Percent | Number of schools Percent 
Total 4,090,015 100 300 100 
Northeast 661,794 16.2 49 16.3 
Midwest 869,419 21.3 63 21.0 
South 1,573,153 38.5 116 38.7 
West 985,649 24.1 72 24.0 


NOTE: Detail may not sum to totals because of rounding. Measure of size is the estimated number of students enrolled in the target grade with a 
minimum of 5 students per school. Region of country is based on Census Bureau definitions. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS), 2015. 


Table 2-25. Number and percentage of students in the sampling frame and number and percentage of 
schools in the sample, U.S. TIMSS fourth grade, by poverty level: 2015 


Frame Sample 
Poverty level Measure of size Percent | Number of schools Percent 
Total 4,090,015 100 300 100 
High 2,022,213 49.4 148 49.3 
Low 2,067,802 50.6 152 50.7 


NOTE: Measure of size is the estimated number of students enrolled in the target grade with a minimum of 5 students per school. For public 
schools, “high” poverty is defined as having 50 percent or more of the students eligible for participation in the free or reduced-price lunch 
program (FRPL), and “low” poverty is defined as having less than 50 percent eligible. Because no FRPL data were available for private schools, 
all private schools are categorized as “low.” 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 


Study (TIMSS), 2015. 


Table 2-26. Number and percentage of students in the sampling frame and number and percentage of 
schools in the sample, U.S. TIMSS fourth grade, by school type: 2015 


Frame Sample 
School type Measure of size Percent | Number of schools Percent 
Total 4,090,015 100 300 100 
Private 350,217 8.6 26 8.7 
Public 3,739,798 91.4 274 91.3 


NOTE: Measure of size is the estimated number of students enrolled in the target grade with a minimum of 5 students per school. 
SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS), 2015. 


ico) 
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Table 2-27. Number and percentage of students in the sampling frame and number and percentage of 
schools in the sample, U.S. TIMSS fourth grade, by locale: 2015 


Frame Sample 
Locale Measure of size Percent Number of schools Percent 
Total 4,090,015 100 300 100 
City 1,287,351 31.5 94 31.3 
Suburb 1,627,468 39.8 118 39.3 
Town 446,097 10.9 32 10.7 
Rural 729,099 17.8 56 18.7 


NOTE: For definitions of these urban-centric locales, see http://nces.ed.gov/surveys/urbaned/definitions.asp. Measure of size is the estimated 
number of students enrolled in the target grade with a minimum of 5 students per school. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS), 2015. 


Table 2-28. Number and percentage of students in the sampling frame and number and percentage of 
schools in the sample, U.S. TIMSS fourth grade, by race/ethnicity status: 2015 


Frame Sample 
Race/ethnicity status Measure of size Percent | Number of schools Percent 
Total 4,090,015 100 300 100 
15 percent or above 3,204,061 78.3 239 79.7 
Below 15 percent 885,954 21.7 61 20.3 


NOTE: Measure of size is the estimated number of students enrolled in the target grade with a minimum of 5 students per school. Race/ethnicity 
status refers to the percentage of Black, Hispanic, Asian, Hawaiian/Pacific Islander, American Indian and Alaska Native students, and multiracial 
students. Black includes African American and Hispanic includes Latino. Racial categories exclude Hispanic origin. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS), 2015. 


Table 2-29. Number and percentage of students in the sampling frame and number and percentage of 
schools in the sample, U.S. TIMSS fourth grade, by region, poverty level, and school type: 


2015 
Frame Sample 
Measure of Number of 
Region Poverty level School type size Percent schools Percent 
Total 4,090,015 100 300 100 
Northeast High Public 228,053 5.6 17 5.7 
Midwest High Public 353,492 8.6 26 8.7 
South High Public 930,446 22.7 68 22.7 
West High Public 510,222 12.5 37 12.3 
Northeast Low Private 77,846 1.9 6 2.0 
Midwest Low Private 88,141 2.2 6 2.0 
South Low Private 116,951 2.9 9 3.0 
West Low Private 67,279 1.6 5 1.7 


See notes at end of table. 
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Table 2-29. Number and percentage of students in the sampling frame and number and percentage of 
schools in the sample, U.S. TIMSS fourth grade, by region, poverty level, and school type: 
2015—Continued 


Frame Sample 
Measure of Number of 
Region Poverty level School type size Percent schools Percent 
Northeast Low Public 355,895 8.7 26 8.7 
Midwest Low Public 427,786 10.5 31 10.3 
South Low Public 525,756 12.9 39 13.0 
West Low Public 408,148 10.0 30 10.0 


NOTE: Detail may not sum to totals because of rounding. Measure of size is the estimated number of students enrolled in the target grade with a 
minimum of 5 students per school. Region of country is based on Census Bureau definitions. For public schools, “high” poverty is defined as 
having 50 percent or more of the students eligible for participation in the free or reduced-price lunch program (FRPL), and “low” poverty is 
defined as having less than 50 percent eligible. Because no FRPL data were available for private schools, all private schools are categorized as 


“low” 
SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 


Study (TIMSS), 2015. 


2.5.2 Eighth-Grade Tabulations 


This section provides an overview of the eighth-grade frame and sample distribution by each of the 
stratification variables. Each table shows the total number and percentage of eighth-grade students in the 
sampling frame (data shown in tables 2-30 through 2-35 and the total number and percentage of schools 
in the TIMSS eighth-grade school sample. By each stratification variable, the tables are census region 
(table 2-30), poverty level (table 2-31), school type (table 2-32), locale (table 2-33), race/ethnicity status 
(table 2-34), and by poverty level, school type, and region (table 2-35). 


Table 2-30. Number and percentage of students in the sampling frame and number and percentage of 
schools in the sample, U.S. TIMSS eighth grade, by region: 2015 


Frame Sample 
Region Measure of size Percent Number of schools Percent 
Total 4,032,863 100 300 100 
Northeast 667,648 16.6 50 16.7 
Midwest 871,091 21.6 65 21.7 
South 1,536,870 38.1 114 38.0 
West 957,254 23.7 71 23.7 


NOTE: Detail may not sum to totals because of rounding. Measure of size is the estimated number of students enrolled in the target grade with a 
minimum of 5 students per school. Region of country is based on Census Bureau definitions. 
SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 


Study (TIMSS), 2015. 
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Table 2-31. Number and percentage of students in the sampling frame and number and percentage of 
schools in the sample, U.S. TIMSS eighth grade, by poverty level: 2015 


Frame Sample 
Poverty level Measure of size Percent | Number of schools Percent 
Total 4,032,863 100 300 100 
High 1,819,876 45.1 135 45.0 
Low 2,212,987 54.9 165 59.0 


NOTE: Measure of size is the estimated number of students enrolled in the target grade with a minimum of 5 students per school. For public 
schools, “high” poverty is defined as having 50 percent or more of the students eligible for participation in the free or reduced-price lunch 
program (FRPL), and “low” poverty is defined as having less than 50 percent eligible. Because no FRPL data were available for private schools, 
all private schools are categorized as “low.” 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS), 2015. 


Table 2-32. Number and percentage of students in the sampling frame and number and percentage of 
schools in the sample, U.S. TIMSS eighth grade, by school type: 2015 


Frame Sample 
School type Measure of size Percent Number of schools Percent 
Total 4,032,863 100 300 100 
Private 340,421 8.4 25 8.3 
Public 3,692,442 91.6 275 91.7 


NOTE: Measure of size is the estimated number of students enrolled in the target grade with a minimum of 5 students per school. 
SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS), 2015. 


Table 2-33. Number and percentage of students in the sampling frame and number and percentage of 
schools in the sample, U.S. TIMSS eighth grade, by locale: 2015 


Frame Sample 
Locale Measure of size Percent Number of schools Percent 
Total 4,032,863 100 300 100 
City 1,221,037 30.3 91 30.3 
Suburb 1,609,501 39.9 118 39.3 
Town 459,329 11.4 36 12.0 
Rural 742,996 18.4 55 18.3 


NOTE: Detail may not sum to totals because of rounding. For definitions of these urban-centric locales, see 
http://nces.ed.gov/surveys/urbaned/definitions.asp. Measure of size is the estimated number of students enrolled in the target grade with a 
minimum of 5 students per school. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS), 2015. 
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Table 2-34. Number and percentage of students in the sampling frame and number and percentage of 
schools in the sample, U.S. TIMSS eighth grade, by race/ethnicity status: 2015 


Frame Sample 
Race/ethnicity status Measure of size Percent | Number of schools Percent 
Total 4,032,863 100 300 100 
15 percent or above 3,120,354 77.4 235 78.3 
Below 15 percent 912,509 22.6 65 21.7 


Ne a 
NOTE: Measure of size is the estimated number of students enrolled in the target grade with a minimum of 5 students per school. Race/ethnicity 
status refers to the percentage of Black, Hispanic, Asian, Hawaiian/Pacific Islander, American Indian and Alaska Native students, and multiracial 
students. Black includes African American and Hispanic includes Latino. Racial categories exclude Hispanic origin. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS), 2015. 


Table 2-35. Number and percentage of students in the sampling frame and number and percentage of 
schools in the sample, U.S. TIMSS eighth grade, by region, poverty level, and school type: 


2015 
Frame Sample 
Measure of Number of 

Region Poverty level School type size Percent schools Percent 

Total 4,032,863 100 300 100 
Northeast High Public 205,600 5.1 15 5.0 
Midwest High Public 304,243 7.5 23 7.7 
South High Public 847,699 21.0 63 21.0 
West High Public 462,334 11.5 34 11.3 
Northeast Low Private 78,479 1.9 6 2.0 
Midwest Low Private 83,266 2.1 6 2.0 
South Low Private 112,910 2.8 8 2.7 
West Low Private 65,766 1.6 5 1.7 
Northeast Low Public 383,569 9.5 29 9.7 
Midwest Low Public 483,582 12.0 36 12.0 
South Low Public 576,261 14.3 43 14.3 
West Low Public 429,154 10.6 32 10.7 


i i | 
NOTE: Detail may not sum to totals because of rounding. Measure of size is the estimated number of students enrolled in the target grade with a 
minimum of 5 students per school. Region of country is based on Census Bureau definitions. For public schools, “high” poverty is defined as 
having 50 percent or more of the students eligible for participation in the free or reduced-price lunch program (FRPL), and “low” poverty is 
defined as having less than 50 percent eligible. Because no FRPL data were available for private schools, all private schools are categorized as 
“low.” 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS), 2015. 


2.5.3 Advanced Tabulations 


This section provides an overview of the advanced frame and sample distribution by each of the 
stratification variables. Each table shows the total number and percentage of twelfth-grade students in the 


(ee) 
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sampling frame (data shown in tables 2-36 through 2-41 and the total number and percentage of schools 
in the TIMSS Advanced school sample. By each stratification variable, the tables are AP status (table 
2-36), census region (table 2-37), school type (table 2-38), locale (table 2-39), race/ethnicity status (table 
2-40), and by AP status, region, and school type (table 2-41). 


Table 2-36. Number and percentage of students in the sampling frame and number and percentage of 
schools in the sample, U.S. TIMSS Advanced, by AP status: 2015 


Frame Sample 
AP Status Measure of size Percent Number of schools Percent 
Total 3,816,235 100 348 100 
AP School 2,942,969 77.1 304 87.4 
Non-AP School 873,266 22.9 44 12.6 


i a an A ae a i a oi a a a a a Ae a a 
NOTE: Measure of size is the estimated number of students enrolled in the target grade with a minimum of 5 students per school. AP status refers 
whether or not the school had students who took a calculus, physics, or both calculus and physics AP test in 2013. The AP schools were 
oversampled. For more information see pages 2-15 through 2-16. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS), 2015. 


Table 2-37. Number and percentage of students in the sampling frame and number and percentage of 
schools in the sample, U.S. TIMSS Advanced, by region: 2015 


Frame Sample 
Region Measure of size Percent Number of schools Percent 
Total 3,816,235 100 348 100 
Northeast 668,328 17.5 60 17.2 
Midwest 838,162 22.0 75 21.6 
South 1,333,932 35.0 123 35.3 
West 975,813 25.6 90 25.9 


NOTE: Detail may not sum to totals because of rounding. Measure of size is the estimated number of students enrolled in the target grade with a 
minimum of 5 students per school. Region of country is based on Census Bureau definitions. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS), 2015. 


Table 2-38. Number and percentage of students in the sampling frame and number and percentage of 
schools in the sample, U.S. TIMSS Advanced, by school type: 2015 


Frame Sample 
School type Measure of size Percent | Number of schools Percent 
Total 3,816,235 100 348 100 
Private 320,657 8.4 29 8.3 
Public 3,495,578 91.6 319 91.7 


NOTE: Measure of size is the estimated number of students enrolled in the target grade with a minimum of 5 students per school. 
SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS), 2015. 
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Table 2-39. Number and percentage of students in the sampling frame and number and percentage of 


schools in the sample, U.S. TIMSS Advanced, by locale: 2015 


Frame Sample 
Measure of size Percent Number of schools Percent 
3,816,235 100 348 100 
1,177,105 30.8 107 30.7 
1,529,268 40.1 148 42.5 
420,122 11.0 36 10.3 
689,740 18.1 57 16.4 


NOTE: Detail may not sum to totals because of rounding. For definitions of these urban-centric locales, see 
http://nces.ed.gov/surveys/urbaned/definitions.asp. Measure of size is the estimated number of students enrolled in the target grade with a 
minimum of 5 students per school. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS), 2015. 


Table 2-40. Number and percentage of students in the sampling frame and number and percentage of 


schools in the sample, U.S. TIMSS Advanced, by race/ethnicity status: 2015 


Frame Sample 
Race/ethnicity status Measure of size Percent | Number of schools Percent 
Total 3,816,235 100 348 100 
15 percent or above 2,930,571 76.8 266 76.4 
Below 15 percent 885,664 23.2 82 23.6 


NOTE: Measure of size is the estimated number of students enrolled in the target grade with a minimum of 5 students per school. Race/ethnicity 
status refers to the percentage of Black, Hispanic, Asian, Hawaiian/Pacific Islander, American Indian and Alaska Native students, and multiracial 
students. Black includes African American and Hispanic includes Latino. Racial categories exclude Hispanic origin. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS), 2015. 


Table 2-41. Number and percentage of students in the sampling frame and number and percentage of 
schools in the sample, U.S. TIMSS Advanced, by AP status, region, and school type: 2015 
Frame Sample 
Measure of Number of 

AP Status Region —_ School type size Percent schools Percent 
Total 3,816,235 100 348 100 

AP school Northeast Public 478,900 12.5 49 14.1 
AP school Midwest Public 552,145 14.5 58 16.7 
AP school South Public 990,211 25.9 104 29.9 
AP school West Public 704,833 18.5 74 21.3 
AP school Northeast Private 59,388 1.6 5 1.4 
AP school Midwest Private 48,075 1.3 5 1.4 
AP school South Private 65,278 1.7 5 1.4 
AP school West Private 44,139 1.2 4 1.1 


See notes at end of table. 
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Table 2-41. Number and percentage of students in the sampling frame and number and percentage of 
schools in the sample, U.S. TIMSS Advanced, by AP status, region, and school type: 
2015—Continued 


Frame Sample 
Measure of Number of 
AP Status Region — School type size _ Percent schools Percent 
Non-AP school Northeast Public 101,947 2.7 4 1.1 
Non-AP school Midwest Public 217,682 5.7 10 2.9 
Non-AP school South Public 238,845 6.3 10 2.9 
Non-AP school West Public 211,015 5.5 10 2.9 
Non-AP school Northeast Private 28,093 0.7 2 0.6 
Non-AP school Midwest Private 20,260 0.5 2 0.6 
Non-AP school South Private 39,598 1.0 4 1.1 
Non-AP school West Private 15,826 0.4 2 0.6 


NOTE: Detail may not sum to totals because of rounding. Measure of size is the estimated number of students enrolled in the target grade with a 
minimum of 5 students per school. AP status refers whether or not the school had students who took a calculus, physics, or both calculus and 
physics AP test in 2013. Region of country is based on Census Bureau definitions. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS), 2015. 


2.6 State School Samples 


The school frame used to draw the state samples was identical to the national frame of public schools in 
Florida.? The rest of the design was similar to the national design where possible. There were only two 
explicit strata per state (high/low poverty level), as the other national strata did not apply to state samples. 
As with the national samples, the frame was implicitly stratified by location, race/ethnicity enrollment, 
and estimated grade enrollment. Tables 2-42 and 2-43 show the number and percentage of students and 
schools included in the Florida TIMSS fourth- and eighth-grade school sampling frame and sample, 
respectively, by poverty level, locale, and race/ethnicity status. The MOS for each school was the same as 
in the national design. Substitute schools were assigned using the same procedure. The procedure for 
selecting classrooms was also the same as in the national design. 


Table 2-42. Number and percentage of students and schools included in the Florida TIMSS fourth-grade 
school sampling frame and sample, by poverty level, locale, and race/ethnicity status: 2015 


Frame Sample 
Poverty Measure of Number of 
level Locale _ Race/ethnicity status size Percent schools Percent 
Total 204,187 100 54 100 
High City 15 percent or above 36,962 18.1 10 18.5 
High City Below 15 percent 0 0.0 0 0.0 


See notes at end of table. 


° For this reason, tables are not provided showing the frame distributions for each benchmarking state by each of the respective stratification 
variables. Tables showing each state’s frame and sample distribution by each of the stratification variables are provided in tables 2-42 and 2-43. 
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Table 2-42. Number and percentage of students and schools included in the Florida TIMSS fourth-grade 
school sampling frame and sample, by poverty level, locale, and race/ethnicity status: 
2015—Continued 


Frame Sample 

Poverty Measure of Number of 

level Locale __Race/ethnicity status size Percent schools Percent 
High Suburb 15 percent or above 78,389 38.4 21 38.9 
High Suburb Below 15 percent 466 0.2 0 0.0 
High Town 15 percent or above 8,131 4.0 2 3.7 
High Town Below 15 percent 238 0.1 0 0.0 
High Rural 15 percent or above 13,150 6.4 4 7.4 
High Rural Below 15 percent 2030 1.0 0 0.0 
Low City 15 percent or above 15,511 7.6 4 7.4 
Low City Below 15 percent 6 0.0 0 0.0 
Low Suburb 15 percent or above 40,503 19.8 11 20.4 
Low Suburb Below 15 percent 1,665 0.8 0 0.0 
Low Town 15 percent or above 1,241 0.6 0 0.0 
Low Town Below 15 percent 0 0.0 0 0.0 
Low Rural 15 percent or above 5,310 2.6 1 1.9 
Low Rural Below 15 percent 585 0.3 1 1.9 


NOTE: Detail may not sum to totals because of rounding. For public schools, “high” poverty is defined as having 50 percent or more of the 
students eligible for participation in the free or reduced-price lunch program (FRPL), and “low” poverty is defined as having less than 50 
percent eligible For definitions of these urban-centric locales, see http://nces.ed.gov/surveys/urbaned/definitions.asp. Race/ethnicity status refers 
to the percentage of Black, Hispanic, Asian and Pacific Islander, and American Indian and Alaska Native students. Black includes African 
American and Hispanic includes Latino. Racial categories exclude Hispanic origin. The measure of size is defined as the estimated number of 
students enrolled in the target grade with a minimum of 5 students per school. These differ slightly from the estimated student sample size for 
fourth grade reported in Methods and Procedures in TIMSS 2015 (Martin, Mullis, and Hooper 2016a), Exhibit 5.5, p. 5.13, which reports the 
estimate of the number of students in the sampled schools with no minimum per school. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS), 2015. 


Table 2-43. Number and percentage of students and schools included in the Florida TIMSS eighth-grade 
school sampling frame and sample, by poverty level, locale, and race/ethnicity status: 2015 


Frame Sample 

Poverty Measure of Number of 
level Locale _Race/ethnicity status size Percent schools Percent 

Total 202,092 100 54 100 
High City 15 percent or above 35,044 17.3 9 16.7 
High City Below 15 percent 0 0.0 0 0.0 
High Suburb 15 percent or above 77,159 38.2 21 38.9 
High Suburb _ Below 15 percent 853 0.4 0 0.0 
High Town 15 percent or above 7,048 3.5 2 3.7 
High Town Below 15 percent 135 0.1 0 0.0 
High Rural 15 percent or above 13,860 6.9 3 5.6 
High Rural Below 15 percent 1,156 0.6 1 1.9 
Low City 15 percent or above 16,223 8.0 4 7.4 
Low City Below 15 percent 0 0.0 0 0.0 
Low Suburb _15 percent or above 40,541 20.1 11 20.4 


See notes at end of table. 
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Table 2-43. Number and percentage of students and schools included in the Florida TIMSS eighth-grade 
school sampling frame and sample, by poverty level, locale, and race/ethnicity status: 
2015—Continued 


Frame Sample 

Poverty Measure of Number of 

level Locale _ Race/ethnicity status size Percent schools Percent 
Low Suburb _ Below 15 percent 1,164 0.6 0.0 
Low Town 15 percent or above 1,216 0.6 1 1.9 
Low Town Below 15 percent 471 0.2 0 0.0 
Low Rural 15 percent or above 6,408 3.2 2 3.7 
Low Rural Below 15 percent 814 0.4 0 0.0 


NOTE: Detail may not sum to totals because of rounding. For public schools, “high” poverty is defined as having 50 percent or more of the 
students eligible for participation in the free or reduced-price lunch program (FRPL), and “low” poverty is defined as having less than 50 percent 
eligible. For definitions of these urban-centric locales, see http://nces.ed.gov/surveys/urbaned/definitions.asp. Race/ethnicity status refers to the 
percentage of Black, Hispanic, Asian and Pacific Islander, and American Indian and Alaska Native students. Black includes African American 
and Hispanic includes Latino. Racial categories exclude Hispanic origin. The measure of size is defined as the estimated number of students 
enrolled in the target grade with a minimum of 5 students per school. These differ slightly from the estimated student sample size for eighth grade 
reported in Methods and Procedures in TIMSS 2015 (Martin, Mullis, and Hooper 2016a), Exhibit 5-6, p. 5.15, which reports the estimate of the 
number of students in the sampled schools with no minimum per school. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS), 2015. 


2.6.1 School Selection for the Florida State Samples 


A TIMSS public school state sample was drawn for Florida at both fourth- and eighth-grade. Ideally, the 
objective for the Florida state samples was that they would not include the schools that were previously 
selected as part of the TIMSS national sample. The Florida state samples were selected using a version of 
the Keyfitz procedure (Keyfitz 1951); Chowdhury, Chu, and Kaufman (2000) have described the 
implementation of the procedure. By again following the Keyfitz procedure outlined in table 2 of 
Chowdhury, Chu, and Kaufman, the procedure allowed us to minimize the overlap with the TIMSS 
national and the NAEP state operational public school sample or “Alpha sample”’® samples. By 
minimizing the overlap with the national sample, the assessed classrooms can be included in only one 
study with proper probabilities. This was accomplished by partitioning the frame into the following four 
groups shown in order as in table 2 of the paper. The four groups were as follows: 


7 public schools not selected for either the TIMSS national or NAEP Alpha samples; 


a public schools selected for the NAEP Alpha sample and not the TIMSS national 
sample; 


5 public schools selected for the TIMSS national sample and not the NAEP Alpha 
sample; and 


7 public schools selected for the TIMSS national sample and the NAEP Alpha sample. 


With this design, the method accomplished the goal of selecting the entire state sample from group 1 and 
none from the other groups. For the schools in groups 3 and 4, this was due to the sum of the school’s 


10 For a complete definition of the NAEP Alpha sample, see the NCES NAEP glossary at 
http://nces.ed.gov/nationsreportcard/glossary.asp#alpha_sample. 
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probabilities of being selected for the state sample and the national sample was always less than one. In 
that case, their conditional probabilities are zero. The Florida state school sample consists of 54 schools at 
both grades four and eight containing a fourth and eighth grade class, respectively. 
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3. PARTICIPATION RATES AND NONRESPONSE BIAS 


To minimize the potential for response biases, IEA developed participation or response rate standards that 
apply to all participating education systems and govern whether or not data are included in the TIMSS 
and TIMSS Advanced 2015 international datasets and the way in which aggregate statistics are presented 
in the international reports. These standards were set using composites of participation rates at the school, 
classroom, and student levels, and were calculated both with and without the inclusion of substitute 
schools (for an explanation of substitute schools, see section 2.4.4) that were selected to replace schools 
refusing to participate. 


The standards take the following three forms, distinguished primarily by whether or not meeting the 
school participation rate of 85 percent requires the counting of substitute schools: 


Category 1: Met requirements. Participants that meet one of the following three sets of conditions are 
considered to have fulfilled the IEA requirements: 


(A) Obtain an unweighted school response rate of at least 85 percent without replacement (rounded to 
nearest whole percent) AND an unweighted student response rate (after rounding) of at least 85 
percent; 


OR 


(B) A weighted school response rate of at least 85 percent without replacement (rounded to nearest 
whole percent) AND a weighted student response rate (after rounding) of at least 85 percent; 


OR 


(C) The product of the (unrounded) weighted school response rate without replacement and the 
(unrounded) weighted student response rate of at least 75 percent (after rounding to the nearest 
whole percent). 


Participants in this category appear in the tables and figures in international reports without annotation, 
and are ordered by achievement score. 


Category 2: Met requirements after substitutes. In the case of participants not meeting the category 1 
requirements, but who had a weighted school response rate of at least 50 percent without replacement 
(after rounding to the nearest percent) AND HAD EITHER: 


(A) A weighted school response rate of at least 85 percent with replacement (after rounding to nearest 
whole percent) AND a weighted student response rate (after rounding) of at least 85 percent; 


OR 
(B) The product of the (unrounded) weighted school response rate with replacement and the 
(unrounded) weighted student response rate of at least 75 percent (after rounding to the nearest 


whole percent). 


Those participants able to satisfy only the category 2 standard are included in the tables and figures but 
are annotated to indicate their response rate status. 
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Category 3: Unacceptable sampling response rate even when substitute schools are included. 
Participants that could provide documentation to show that they complied with TIMSS and TIMSS 
Advanced sampling procedures and requirements but did not meet the requirements for Category 1 or 
Category 2 were be placed in Category 3. Participants in this category appeared in a separate section of 
the achievement tables, below the other participants, in international reports. These countries were 
presented in alphabetical order. 


3.1 TIMSS and TIMSS Advanced Participation Rates of U.S. 
Schools, Classrooms, and Students 


The raw numbers on which the various participation rates are based, along with the participation rates 
themselves, are shown in table 3-1 separately for the TIMSS fourth-grade and eighth-grade national 
samples, table 3-2 for the TIMSS Advanced mathematics and physics national samples, and table 3-3 
separately for the TIMSS fourth-grade and eighth-grade Florida samples. 


To explain how to interpret these participation rates, subsections 3.1.1 through 3.1.3 describe a complete 


interpretation of the TIMSS fourth-grade numbers. The participation rates for TIMSS eighth-grade, 
TIMSS Advanced, and all the state samples can be interpreted in the same way. 
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Table 3-1. Number of U.S. schools, classrooms, and students participating in TIMSS, and participation 
rates, by grade: 2015 


Participation rates 


Number Grade 4 Grade 8 
Item Grade 4. Grade 8 Unweighted Weighted Unweighted Weighted 
Schools 
Sampled 300 300 T T T T 
Excluded and ineligible 5 7 T T T T 
Eligible 295 293 T T T T 
Participating 228 229 77 77 78 78 
Substitutes 22 17 T T T T 
Participating (all schools) 250 246 85 85 84 84 
Classrooms in participating schools 
Total 1,118 2,878 T T T T 
Excluded 33 69 T T T T 
Eligible 1,085 2,809 tT T T T 
Sampled 488 493 T T T T 
Participating 487 490 100 100 99 99 
Students in participating schools 
Sampled 11,267 11,489 T T T T 
Excluded 648 439 T T T T 
Withdrawn 147 198 tT tT tT tT 
Eligible 10,472 10,852 T T T T 
Absent 4431 6312 T tT T T 
Assessed 10,029 10,221 96 96 94 94 


+ Not applicable. 

' Includes 68 students whose parents denied permission to participate. 

? Includes 124 students whose parents denied permission to participate. 

NOTE: NCES standards (Standard 1-3-8) indicate that participation rates should be calculated without including substitute schools since 
substitute schools do not have an independent probability of selection (National Center for Education Statistics 2012). However, the participation 
rates shown in this table are those reported by TIMSS and do include substitute schools in the calculations. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS), 2015. 
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Table 3-2. Number of U.S. schools and students participating in TIMSS Advanced, and participation 
rates, by subject: 2015 


Participation rates 


Advanced 
Number mathematics Physics 
Advanced 
Item mathematics Physics |Unweighted Weighted Unweighted Weighted 
Schools 
Sampled 348 348 T T T T 
Excluded and ineligible 32 111 T T T T 
Eligible 316 237 T T T T 
Participating 230 156 73 72 66 65 
Substitutes 11 9 tT T T T 
Participating (all schools) 241 165 76 76 70 68 
Students in participating schools 
Sampled 3,488 3,539 T T T T 
Excluded 2 6 T T T tT 
Withdrawn 57 114 T T tT tT 
Eligible 3,429 3,419 T T T T 
Absent 475! 4872 tT T T T 
Assessed 2,954 2,932 86 87 86 85 


+ Not applicable. 

' Includes 268 students whose parents denied permission to participate. 

? Includes 255 students whose parents denied permission to participate. 

NOTE: NCES standards (Standard 1-3-8) indicate that participation rates should be calculated without including substitute schools since 
substitute schools do not have an independent probability of selection (National Center for Education Statistics 2012). However, the participation 
rates shown in this table are those reported by TIMSS Advanced and do include substitute schools in the calculations. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), 2015, Trends in International Mathematics and 
Science Study (TIMSS), 2015. 
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Table 3-3. Number of U.S. schools, classrooms, and students participating in TIMSS Florida, and 
participation rates, by grade: 2015 


Participation rates 


Number Grade 4 Grade 8 
Item Grade 4__ Grade 8 Unweighted Weighted Unweighted Weighted 
Schools 
Sampled 54 54 T T T T 
Excluded and ineligible 1 0 T T T T 
Eligible 53 54 T T T T 
Participating 53 53 100 100 98 98 
Substitutes 0 0 T T T T 
Participating (all schools) 53 53 100 100 98 98 
Classrooms in participating schools 
Total 299 940 T T T T 
Excluded 11 9 T T T T 
Eligible 288 931 T T T T 
Sampled 106 106 T T T T 
Participating 106 105 100 100 99 99 
Students in participating schools 
Sampled 2,269 2,336 T T T T 
Excluded 76 47 P T T T 
Withdrawn 55 38 T T T T 
Eligible 2,138 2,251 tT tT T T 
Absent 113! 177° T T tT T 
Assessed 2,025 2,074 95 95 92 93 


+ Not applicable. 

' Includes 9 students whose parents denied permission to participate. 

? Includes 26 students whose parents denied permission to participate. 

NOTE: NCES standards (Standard 1-3-8) indicate that participation rates should be calculated without including substitute schools since 
substitute schools do not have an independent probability of selection (National Center for Education Statistics 2012). However, the participation 
rates shown in this table are those reported by TIMSS and do include substitute schools in the calculations. 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS), 2015. 


3.1.1 Interpreting School Participation Rates (TIMSS Fourth-Grade Example) 


The fourth-grade school sample consisted of 300 schools and was designed to yield a representative 
school sample for TIMSS. Five ineligible schools were identified on the basis that they served special 
student populations or had closed or altered their grade makeup since the sampling frame was developed. 
This left 295 eligible schools, of which 228 agreed to participate. The fourth-grade school participation 
rate before substitution was 77 percent (unweighted). The analogous weighted school participation rate 
was also 77 percent. 


In addition to the 228 participating schools from the original sample, 22 substitute schools also 


participated for a total of 250 participating schools at the fourth grade in the United States. This gave a 
weighted (and unweighted) school participation rate after substitution of 85 percent. 
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3.1.2 Interpreting Classroom Participation Rates (TIMSS Fourth-Grade 
Example) 


In accord with the international requirements, schools agreeing to participate in TIMSS at grade 4 were 
asked to list their fourth-grade mathematics classes as the basis for sampling at the classroom level. 
Schools appeared to be able to identify classes in this way without any problems. A total of 1,118 
mathematics classrooms were identified as a result. At this time, schools were given the opportunity to 
identify special classes—classes containing all, or a majority of, students with intellectual and/or 
functional disabilities, or students who were non-native-language speakers. While these classes were 
regarded as eligible, the students as a group were treated as excluded since, in the opinion of the school, 
their disabilities or language capabilities would render meaningless their performance on the assessment. 
A total of 33 classrooms were excluded in this way. This left a pool of 1,085 eligible classrooms from 
which the sample was drawn. While the students in these excluded classrooms did not figure in the 
participation rate calculations, they did count in the population coverage calculations, and this is reflected 
in the higher exclusion rate for the United States. In the international report the United States is annotated 
to reflect this fact. 


Classrooms with fewer than 15 students were collapsed into “pseudoclassrooms” prior to sampling so that 
each eligible classroom in a school had at least 20 students. Two classrooms (regardless of whether 
regular classrooms or pseudoclassrooms) were selected per school where possible. In schools where there 
was only one classroom, this classroom was selected with certainty. Some 488 classrooms were selected 
as aresult of this process, and 487 participated in TIMSS. Weighted and unweighted classroom 
participation rates were 100 percent. 


Subsequently, schools were asked to list the students in each of the 488 sampled classrooms at the fourth 
grade, along with the teachers who taught mathematics and science to these students. At this time, schools 
were given the opportunity to identify particular students not suited to take the test because of functional 
and/or intellectual disabilities and/or because they were non-native-language speakers (students with 
disabilities or non-native-language speakers who had been mainstreamed; see definitions in section 3.3). 


3.1.3 Interpreting Student Participation Rates (TIMSS Fourth-Grade Example) 


A total of 11,267 fourth-grade students were listed as being in these classrooms. (In mixed-grade 
classrooms only students in the target population were considered.) At the outset, 648 of these were 
excluded because of functional or intellectual disabilities or because they were non-native-language- 
speakers. Additionally, in the months between the listing of students and the time of the assessment, 147 
students were classified as withdrawn, as they were no longer in the school/classroom at the time of the 
assessment. As a consequence, 10,472 students were considered eligible to take the assessment. On the 
day of the assessment some 443 students were absent, leaving 10,029 students who completed a TIMSS 
2015 assessment booklet. Participation rates were calculated on the number of eligible students (10,472). 
Since 10,029 of the 10,472 eligible students were assessed, the weighted (and unweighted) student 
participation rate was 96 percent. 
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3.1.4 Combined Participation Rates 


The combined school, classroom, and student weighted participation rate standard of 75 percent used by 
TIMSS in situations in which it was necessary to recruit substitute schools was met for the grade 4 and 8 
samples in TIMSS 2015. The weighted product of the separate participation rates for TIMSS at grade 4 
was 81 percent (also unweighted) and at grade 8 was 79 (78 percent unweighted). The application of 
international guidelines means, however, that U.S. statistics describing fourth- and eighth-grade students 
in TIMSS are annotated in international reports to indicate that coverage of the defined student population 
was less than the IEA standard of 95 percent (see section 3.3.3 for details on coverage standards) and that 
participation rates were met only after substitute schools were included. 


The weighted product of the separate participation rates for TIMSS Advanced mathematics was 66 
percent (also 66 percent unweighted) and physics was 58 percent (60 percent unweighted). 


The Florida grade 4 and 8 samples both met the participation requirements for category 1 above and are 
considered to have fulfilled the IEA requirements. The weighted and unweighted product of the separate 
participation rates for Florida grade 4 was 90 percent and grade 8 was 95 percent. 


3.2 Participation Rates for All Countries 


For comparable fourth- and eighth-grade school, classroom, and student participation rates in other 
nations in TIMSS, see exhibits C-3 through C-8 in TIMSS 2015 International Results in Mathematics 
(Mullis et al. 2016a) and in TIMSS 2015 International Results in Science (Martin et. al. 2016). For 
comparable participation rates in other nations in TIMSS Advanced, see exhibits C-3 through C-5 in 
appendix C in TIMSS Advanced 2015 International Results in Advanced Mathematics and Physics 
(Mullis et al. 2016b). 


3.3 Exclusions 


The nationally defined target population is described in Chapter 3 of Methods and Procedures in TIMSS 
2015 (Martin, Mullis, and Hooper 2016a) and Methods and Procedures in TIMSS Advanced 2015 
(Martin, Mullis, and Hooper 2016b). All schools and students excluded from this population are referred 
to as the “excluded population.” Exclusions could occur at the school level, with entire schools being 
excluded, or within schools, with specific students or entire classrooms excluded. 


3.3.1 School Exclusions 


Countries could exclude schools that 
7 were geographically inaccessible; 


rT were of extremely small size; 
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a offered a curriculum or school structure radically different from the mainstream 
educational system; or 


a provided instruction only to students in the excluded categories defined under “within- 
school exclusions,” such as schools for the blind. 


3.3.2 Within-School Exclusions 


Countries were asked to adapt the following international within-school exclusion rules to define 
excluded students: 


Students with intellectual disabilities. Students who, in the professional opinion of the school principal 
or other qualified staff members, were considered to be intellectually disabled or who had been tested 
psychologically as such. This included students who were emotionally or mentally unable to follow even 
the general instructions of the test. Students were not to be excluded solely because of poor academic 
performance or normal disciplinary problems. It should be noted that students with dyslexia, or other such 
learning disabilities, should be accommodated in the test situation if possible, rather than excluded. 


Students with functional disabilities. Students who were permanently physically disabled in such a way 
that they could not perform in the TIMSS or TIMSS Advanced testing situation. Functionally disabled 
students who were able to respond were included in the testing. 


Non-native-language speakers. Students who were unable to read or speak the language(s) of the test 
and were unable to overcome the language barrier of the test. Typically, a student who had received less 
than 1 year of instruction in the language(s) of the test was excluded. 


3.3.3 Exclusions in the U.S. National Samples 


As noted earlier, schools were given the opportunity to exclude any special classes among the total 
number of classes in the fourth or eighth grade. These classes were made up largely of students with 
functional or intellectual disabilities or students who were non-native-language speakers, as defined 
above. Classes identified in this way were excluded from the class sampling procedure. Subsequently, 
schools were given the opportunity to exclude students from the sampled classes—essentially, students 
with functional or intellectual disabilities, or non-native-language speaking students in the United States 
who had been mainstreamed. Nevertheless, students with disabilities and/or English language learners 
were allowed access to many accommodations that they received on their state assessments. 


These procedures resulted in a (weighted) student exclusion rate of 6.8 percent in the fourth grade for 
TIMSS, and 5.1 percent for TIMSS in the eighth grade based on the combination of whole-class and 
within-class exclusions. IEA standards define this degree of coverage of the national target population (93 
and 95 percent for TIMSS at fourth and eighth grade, respectively) as acceptable although falling below 
the desired range of 95 percent or better for the fourth grade. The tabulations shown in the international 
reports show the United States fourth grade annotated to indicate this fact. These procedures also resulted 
in a (weighted) student exclusion rate of 0.1 percent in the TIMSS Advanced mathematics and 0.1 percent 
in the TIMSS Advanced physics. IEA standards define this degree of coverage of the target populations 
as acceptable. Lastly, these procedures resulted in a (weighted) student exclusion rate of 4.7 percent in the 
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fourth grade for Florida and 2.8 percent in the eighth grade for Florida. IEA standards define this degree 
of coverage of the target populations as acceptable. 


3.4 Nonresponse Bias Analysis 


The National Center for Education Statistics (NCES) standards for surveys stipulate that a nonresponse 
bias analysis is required at any stage of data collection with a weighted unit response rate of less than 85 
percent (without substitution). Because the U.S. TIMSS 2015 weighted school response rates at grade 4, 
grade 8, Advanced mathematics, and Advanced physics are below 85 percent, NCES required an 
investigation into the potential magnitude of nonresponse bias at the school level in the U.S. samples, 
which is the focus of this section. The Florida samples did not require that a nonresponse bias analysis be 
performed because their weighted school and student participation rates were above 85 percent. 


A bias analysis was conducted in the United States to address potential problems in the data owing to 
school nonresponse for TIMSS grades 4 and 8 and for both subjects in TIMSS Advanced. The purpose of 
the analysis was to examine whether the participation status of schools was related to various 
characteristics and thus introduced the potential for bias in the results. The results suggested that there is 
some potential for nonresponse bias in the U.S. grades 4 and 8 samples (prior to substitution) based on the 
characteristics studied. It also suggested that, while there was some evidence that the use of substitute 
schools at grade 4 reduced the potential for bias, it did not reduce it substantially. At grade 8, the use of 
substitute schools did not reduce the potential for bias, nor did it add to it substantially. However, after the 
application of school nonresponse adjustments, there was no evidence of resulting potential bias in the 
final sample in either grade. 


Analyses of TIMSS Advanced suggest that there is little potential for nonresponse bias in the advanced 
mathematics sample based on the characteristics studied. It also suggests that, while there is some 
evidence that the use of substitute schools has not reduced the potential for bias, it has not added to it 
substantially. Moreover, after the application of school nonresponse adjustments, there is little evidence 
of resulting potential bias in the final sample. In physics, however, the results suggest that there is some 
potential for nonresponse bias in the sample based on the characteristics studied. It also suggests that, 
while there is some evidence that the use of substitute schools reduced the potential for bias, it has not 
reduced it substantially. Moreover, after the application of school nonresponse adjustments, there is some 
evidence of resulting potential bias in the final sample with the largest bias in the school-level 
characteristic locale. 


The full text of the nonresponse bias analysis conducted for TIMSS and TIMSS Advanced 2015 can be 
found in appendixes F and G, respectively. 
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4. SURVEY OPERATIONS 


This chapter describes data collection and related activities for TIMSS and TIMSS Advanced 2015 in the 
United States. These activities included recruitment of schools for the national and Florida samples; 
sampling of students within schools; development of the instruments; field operations undertaken to 
administer the assessment; post-assessment activities associated with scoring and data entry; and several 
activities associated with the preparation of the data to meet international standards. 


4.1 Recruiting Districts and Schools 


The established protocol for seeking the participation of schools in studies such as TIMSS and TIMSS 
Advanced, where participation is voluntary, is to (1) notify state education authorities that they have 
districts and schools selected for the assessment, (2) inform authorities at the district level that schools 
within their districts are being sampled, and (3) contact the sampled schools. For public schools, steps (2) 
and (3) were conducted by NAEP State Coordinators in each state education agency. Participation may be 
refused at any of these levels, so several considerations were important in this context, specifically the 
need to establish the value of participation; establish the timing of the assessment window in conjunction 
with mandatory federal, state, and local assessments; and address concerns about the burden on schools. 
Private schools, including nonreligious affiliated private schools, were contacted directly. In the case of 
Catholic schools, the diocese was informed and schools were then contacted. 


4.1.1 Timing of Recruitment Activities 


Assessment dates needed to be established early in the school year in which the assessment was to take 
place or, better still, toward the end of the previous school year. In total, the recruitment phase for TIMSS 
and TIMSS Advanced extended from April 2014 through May 2015, as indicated in exhibit 1-1 in 

chapter 1. States were contacted toward the end of April 2014. Following this, contact with districts began 
in May 2014 and continued through the summer. Schools were contacted beginning in June 2014 and 
activities continued through May 2015. 


4.1.2 The Impact of the Sampling Design on Recruitment Activities 


The sampling design played an important role in the design of recruitment activities. For TIMSS, at both 
grade levels, 300 schools were sampled in the first instance, along with 300 first substitutes and 300 
second substitutes, for a total of 900 schools at each grade level. For TIMSS Advanced at grade 12, there 
were 348 schools in the original sample, along with 348 first substitutes and 348 second substitutes, for a 
total of 1,044 high schools. In gaining the cooperation of schools, at each grade level the sampled schools 
were approached in the first instance. Operationally this meant first informing the districts in which these 
schools are located and then approaching the schools themselves. If a sampled school refused to 
participate, the district of the first substitute school was approached and the district-school permission 
procedure began anew. If the first substitute school refused as well, then the district of the second 
substitute school was approached, and then the second substitute school. 
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4.1.3 Contacting States and Districts 


The chief state school officer and state assessment director in each of the 50 states and Washington, DC 
were contacted beginning in April 2014. Each person received a combined TIMSS and TIMSS Advanced 
package that included an NCES cover letter, instructions on how to obtain the sampled schools in the 
state, a brochure describing the study, a timeline of activities, a summary of activities for the school 
coordinator, and a sheet of frequently asked questions. A copy of the letter sent to states is provided as 
exhibit A-1 in appendix A. Several items of TIMSS and TIMSS Advanced information materials (exhibits 
B-1 through B-13 in appendix B) were included with the letter. 


Similar packages were supplied to NAEP State Coordinators. NCES and Westat held a series of webinars 
to explain TIMSS and TIMSS Advanced, to describe the processes used to report participation, and to 
answer questions about the study. In many cases, NAEP State Coordinators created personalized contact 
letters for districts and schools. Westat tracked cooperation status weekly through a secure automated 
reporting system used by NAEP State Coordinators. 


After informing the states, similar packages of recruitment materials were sent to the superintendent and 
the assessment director of each district by the NAEP State Coordinator (or diocese by Westat field staff) 
containing sampled schools. A copy of the letter sent to districts is provided as exhibit A-2 in appendix A. 
Several items of TIMSS and TIMSS Advanced information materials (exhibits B-1 through B-13 in 
appendix B) and a list of sampled schools were included with the letter. 


During this time, if a sampled school in a cooperating district refused to participate and was judged a firm 
refusal, a similar district package was sent out to the district of the first substitute school linked to the 
sampled school. A parallel procedure was adopted with the second substitute district and school in those 
cases where a first substitute school refused to participate. 


Follow-Up Contact 


In each case, NAEP State Coordinators or Westat field staff were available to district contacts to discuss 
the study and answer questions. 


Special security requirements. As a matter of course, each TIMSS and TIMSS Advanced field staff 
member had a current FBI clearance and fingerprints on file. In addition, all field staff had eQIP clearance 
as well. These requirements satisfied school security requirements. 


4.1.4 Contacting Schools 


After district approval was secured, schools were contacted with an initial school information packet. 
Private schools and some parochial schools not linked with a diocese were contacted directly. Each school 
information package was sent by the NAEP State Coordinator on a flow basis governed by receipt of 
district approval. A copy of the cover letter included in the school package is shown as exhibit A-3 and 
A-4 in appendix A. Copies of TIMSS and TIMSS Advanced information materials (exhibits B-1 through 
B-13 in appendix B) were included with the letter. 
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Follow-Up Contact 


After a few days, for private schools, each school was contacted by a Westat field staff member to discuss 
the school’s participation in TIMSS. In-person visits were made to a small number of private schools to 
enhance efforts to secure participation. NAEP State Coordinators continued follow-up with public schools 
when necessary. 


Nomination of school coordinator. School principals were asked to identify an individual within the 
school who would act as the TIMSS or TIMSS Advanced school coordinator for their school. Principals, 
deputy principals, teachers, test coordinators, or guidance counselors took on this role. School coordinator 
responsibilities included the following: 


7 working with Westat to finalize an assessment date; 
cs preparing an electronic list of all classrooms in grade 4 and mathematics classrooms in 
grade 8; 


a preparing an electronic list of all students eligible for TIMSS Advanced at grade 12; 


. identifying excluded classrooms; 
7 identifying students and teachers in the sampled classrooms at grade 4 and 8; 
: identifying excluded students in the sampled classrooms at grade 4 and 8 and among 


sampled students for TIMSS Advanced; 


7 liaison with staff, students, and parents as necessary; 

. arranging for space and for the release of students from classes on assessment day; 
a ensuring completion and return of the school and teacher questionnaires; and 

a holding secure, until the end of the school year, the confidential files that linked 


student names with IDs and then destroying them. 


4.1.5 Informational Materials and Gifts 


Given that most of the initial contact with states, districts, and schools was by mail, particular attention 
was paid to developing materials that would promote the value of participation and assure all concerned 
that the burden on schools would be minimal. These materials included the following: 


a TIMSS and TIMSS Advanced brochures, a tri-fold describing TIMSS or TIMSS 
Advanced, its history, and its importance for the United States (exhibits B-1 and B-4 
in appendix B); 


. TIMSS grade 4 and grade 8, and TIMSS Advanced Frequently Asked Questions 
sheets (exhibits B-2 and B-5 in appendix B); 
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5 Summary of School Activities listing broad timeframes and activities occurring within 
them for TIMSS at grade 4 and grade 8, and TIMSS Advanced (exhibit B-3 and B-6 
in appendix B); and 


rT TIMSS and TIMSS Advanced Class and Student List Submission Tip sheets (exhibits 
B-7 through B-13). 


After the assessment, schools and school coordinators were paid incentives of $200 and $100, 
respectively. Students each received a carabiner with a mini-flashlight attached. 


Some additional documents were used to maintain contact with schools throughout the year and to inform 
school coordinators of coming TIMSS and TIMSS Advanced activities. A MyTIMSS website was 
launched in early 2015 that shared information about school coordinator responsibilities and other 
information in preparation for assessment day. Professional development webinars were conducted on 
various topics in large-scale international assessments that school staff could attend, and for which they 
could receive certificates of attendance for possible use in maintaining their local certification. Drafts of 
parent notification letters were available to all schools, and parent approval letter, forms, and fact sheets 
were supplied to those schools indicating that parent approval was required. Examples of the parent 
approval materials are provided in exhibits A-6 through A-22 in appendix A. The parent 
notification/approval process is discussed further below. 


4.1.6 Gaining Cooperation Recruiters 


Private schools at grades 4, 8 and 12 were recruited by two experienced Westat recruiters known as 
“gaining cooperation recruiters” (GCRs), who were experienced with recruiting schools from previous 
rounds of similar studies (NAEP, PISA, TIMSS, and PIRLS), and a GCR field manager. 


GCR field manager responsibilities. The field manager engaged in the recruiting phase of the study had 
the following responsibilities: 


7 coordinating recruitment activities of the GCRs within their assignment; 


7 holding weekly one-on-one telephone meetings with their GCRs to monitor progress 
on gaining-cooperation activities and to troubleshoot recruitment strategies; 


: monitoring and maintaining the Field Management System, ensuring that the 
disposition codes provided by GCRs gave an accurate portrayal of participation status; 
and 

u acting as troubleshooters to handle special issues that arose during the recruiting 
process. 


GCR responsibilities. The primary responsibility of the two GCRs was person-to-person interaction with 
the schools along with some more administrative tasks, as follows: 


a making telephone contact to sampled schools, and in some cases, their diocese, within 
their assignment to obtain permission to conduct TIMSS and TIMSS Advanced; 


7 making in-person refusal conversion attempts when necessary and requested by a field 
manager; 
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5 completing an electronic record of calls (EROC) of each contact within the MyTIMSS 
School Control System, 


a maintaining the most current disposition codes within the MyTIMSS School Control 
System; and 
7 meeting weekly with the field manager by conference call to discuss progress. 
4.1.7 Monitoring the Recruiting Progress 


Progress in recruiting districts and schools was monitored by Westat through the MyTIMSS School 
Control System for public schools being recruited by NAEP State Coordinators. MyTIMSS is a secure 
password-protected website, with access restricted to NAEP State Coordinators and Westat home office 
staff. NAEP State Coordinators were required to update the MyTIMSS School Control System for each 
contact made with districts or schools using an electronic record of calls that included updating the 
disposition code for the district/school in question. The disposition codes in the MyTIMSS School 
Control System indicated whether the school was pending, refusing, or cooperating. Using these status 
codes, the Westat home office tracked the progress of recruitment and generated weekly reports. These 
weekly reports enabled the operations of recruitment (and eventually assessments) to be closely 
monitored. 


Difficulties in Gaining Cooperation 


The principal reasons given by both districts and schools for refusing to participate included, in 
approximate order of occurrence, the following: 


: conflict with mandatory federal, state, and/or local assessments whose outcomes had 
direct implications for districts, schools, teachers and students, and with AP testing for 
TIMSS Advanced, with direct implications for students; 


5 the related matter of the burden that additional testing placed on students at the cost of 
instructional time; 


5 concerns about too much testing and parents opting out of assessments; and 


a the limited return on the school’s investment of time because they would not receive 
much usable information on the school, and none on particular students. 


4.2 Sampling Students Within Schools 


For TIMSS 2015, grade 4 and grade 8 students in participating schools were sampled in a two-stage 
process. In the first stage, schools were asked to provide lists of fourth-grade classrooms or eighth-grade 
mathematics classrooms that indicated the number of students in each class. An equal probability sample 
of two classrooms (or pseudoclassrooms) was identified from the classrooms listed for each school. A 
pseudoclass is two or more classes that were too small to be assessed alone but were combined with other 
small classes in the sampling software. In the second stage all students in sampled classrooms (or 
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pseudoclassrooms) were selected for assessment. For TIMSS Advanced 2015, similarly to TIMSS 2015, a 
two-stage sampling process was conducted with the first stage a sample of schools, and the second stage a 
sample of advanced students within schools rather than classrooms. Detailed descriptions of sampling 
designs and methods are covered in chapter 2. 


These procedures are standardized internationally and embodied in software made available to each 
country. The software was developed by the International Association for the Evaluation of Educational 
Achievement (IEA) Data Processing Center and is known as IEA Windows Within School Sampling 
Software, or WinW3S (IEA Data Processing Center 2014). The WinW3S system provides for forms 
generation, data entry, class sampling, student sampling, student-teacher linkages, the random assignment 
of assessment booklets to students, the production of various survey tracking forms, and the printing of 
labels for test instruments and questionnaires. 


Westat home office staff attended a WinW3SS training session led by IEA. In a departure from previous 
administrations of TIMSS, Westat developed an E-File system to integrate the data collection with the 
WinWs3sS sampling system. This eliminated the hard-copy collection of class and student data and the 
requisite resulting data entry and cumbersome back and forth with schools. The staff assigned to process 
the electronic data collection reviewed the incoming data, clarified any issues or inconsistencies with the 
schools (via school coordinators), properly formatted and imported the data into the WinW3S, and 
executed the sampling procedures within the WinW3S system. The staff responsible for the sampling 
processing also generated the materials (all of the forms and labels) needed by field staff and schools for 
the TIMSS assessment. All Westat staff handling PII signed nondisclosure affidavits, and all PII were 
kept secure on Westat secure servers. 


4.2.1 Obtaining Electronic Class Lists From Schools at Grades 4 and 8 


As previously noted, the data collection for the sampling information from schools was, for the first time 
in TIMSS, conducted electronically. In preparation for the TIMSS and TIMSS Advanced 2015 
assessment period, Westat coordinated with school staff and field staff to set up secure, password- 
protected file transfer protocols between schools and Westat. The implementation of the secure TIMSS 
E-File system eliminated the tedious and difficult task of converting hard-copy information into the 
electronic data within the WinW3S. Another benefit of the new E-File system was that seamless 
automated electronic receipting and data tracking were incorporated into the data collection and 
administrative operation, which was not possible in previous TIMSS administrations. 


School coordinators worked with schools to collect, review, and provide Westat with the data that would 
match what had been previously captured in the hard-copy forms. The treatment for TIMSS (grades 4 and 
8) was handled the same as for TIMSS Advanced although the data collected took a different path. For 
TIMSS, all classes from the schools with grade 4 and grade 8 students were requested along with the list 
of all students in those classes (see section 4.2.2). For TIMSS Advanced, Westat worked with the schools 
and/or school coordinators to pre-select the courses that met the criteria to be eligible for the study. 
Westat then requested the list of all twelfth-grade students who had taken or were taking the eligible 
classes regardless of whether the class was taken in grade 11 or grade 12. Therefore, there are no Class 
Listing Forms for TIMSS Advanced. 


For TIMSS, an EXCEL template with the data elements requested in the Class Listing Form (CLF) was 
transmitted to the school coordinator. The CLF template was used to create a list of the eligible classes, 
some attributes of each class, and the names of the teacher(s) teaching each class. An illustrative Class 
Listing Form template for grade 4 is shown in exhibit 4-1 for a small school. Exhibit 4-2 illustrates the 
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class listing form template for a larger grade 4 school that has a different science teacher than the 
mathematics teacher. A Class Listing Form for grade 8 is shown in exhibit 4-3 that shows the students are 
linked only to their mathematics class and teacher. In order to properly link these students to a science 
teacher and science class, the schools submitted a teacher list (as seen in exhibit 4-4) that identified all of 
the science teachers and the classes they taught. Later on in the process, this information was used to 
generate a Student-Teacher-Linking Form that was sent to the schools to indicate which science 
teachers/classes the student was taking. 


The templates provided to the schools allowed for some flexibility in how the schools provided the data. 
For example, if the school information system stored students’ month and year of birth (MOB and YOB) 
in a single field, or in multiple fields, Westat provided alternative templates for them to use. Westat 
wanted to minimize the burden for schools in providing the requisite data. Often schools could provide 
much of the data requested in an automated way by pulling the data directly from their school information 
systems. By eliminating much of the manual entry by schools, and by providing flexibility even when 
manual entry was necessary, the quality and speed of obtaining data from schools was enhanced. That 
allowed TIMSS data collection and sampling to take place closer to the time of assessment, which 
provided for a more stable sample of classes and students in these classes than if samples were provided 
earlier in the school year. 


Since the E-File system was set up to communicate with each school, the administrative information 
(school name, TIMSS-generated ID, and grade assessed) was already pre-loaded into the data request 
Westat would send and receive from the schools. Schools were asked to complete the remaining 
information for each eligible fourth-grade class or eighth-grade mathematics class in the school. 


Exhibit 4-1. Sample Class Listing Form template (grade 4) for a small school with the same 
mathematics and science teacher for each class 


Please refer to important definitions and examples by hovering over the @ in each column header. These definitions are also provided in the Class Roster Definitions. 


Use the Save button to save information you have entered for each class. 

If you need to delete a class, click on the @ at the beginning of a saved class row. 

Use the Print button to print a list of the classes you included for reference. 

Click on the Finished button when you have entered all fourth-grade classes in the form. 


Grade 4 


=. es Number @ class _ : 
\U' Class Name \U’ Class Grou of Exclusion \U’ Name of Mathematics Teacher \U’ Name of Science Teacher 
Students Status (If applicable) 
¥ 


Select.. v Select... 


When you choose "Submit Class List" for Grade 4, you 
will see this empty form. 


Save Print Finished 


Grade 4 


= ss umber @ Class = z 
\U’ Class Name \' Class Group of Exclusion \L' Name of Mathematics Teacher \t' Name of Science Teacher 
Students Status (If applicable) 
30 


Students with intellectual disabilities » ||Anne Jensen Anne Jensen 


Jensen Average v 


29 Select... bg | Trevor Adams | Trevor Adams 


Adams 


Choose "Finished" once you have entered all class information for 
Save Print Grade 4. No further editing is possible when you choose this option. 
SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS), 2015. 


High ov 
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Exhibit 4-2. Sample Class Listing Form template for a larger school with multiple grade 4 classes 


Number @ Class f 
of Exclusion ‘\L' Name of Mathematics Teacher ame o 2 
Students Status (If applicable) 


Grade 4-1 Average ¥ 26 | Students with intellectual disabilities v ||Paul Smith Julia O'Donnell 


Grade 4 


Grade 4-2 High ’ | Select... y ||Mary Porter Julia O'Donnell 


|Grade 4-3 Low ba | Students with intellectual disabilities y ||William Peters Julia O'Donnell 


Save Print | [ Finished || Choose "Finished" only when you have entered all classes. No further editing is possible once you choose this option. 


SOURCE: National Center for Education Statistics (NCES), Trends in International Mathematics and Science Study (TIMSS), 2015. 


Exhibit 4-3. Sample Class Listing Form template for grade 8 


Number @ Class 
of Exclusion ame of Mathematics Teacher 
Students Status (If applicable) 


Students with intellectual disabilities ~ 


Grade 8 


Charles Moore 


Click "Save" to save a class and create an empty row for another class. When you are finished, move on 
to the science teacher section, below. 


Students with intellectual disabilities »~ ||Rebecca Newman 


Select... y ||Rebecca Newman 


Intro to Algebra Select... Charles Moore 


Save 


SOURCE: National Center for Education Statistics (NCES), Trends in International Mathematics and Science Study (TIMSS), 2015. 


TIMSS 2015 Grade 4: Class Roster Definitions 


L_|Class name. Record the class name that is typically used by your school to refer to the class. For 
example, it may be that your school uses the grade plus a letter for the class name (4a, 4b, etc.), the grade 
plus a number (4.1, 4.2, etc.), the class location (Room 7, Room 8, etc.,) or some other scheme. This is 
important because these identifiers will be used to indicate to the Test Administrator which classes 
will be tested. 


L]Class group or track (if applicable). If your school assigns students to specific classes based on their 
ability, please indicate the relevant level: Low ability, Average ability, or High ability. 


L|Number of students. Enter the number of fourth-grade students in each class. In the case of multigrade 
classes (e.g., students from more than one grade level in the same class), only the fourth-grade students 
should be counted as a class in the list. For example, if three grade 3 students, five grade 4 students, and 
ten grade 5 students form a multigrade class, then you should record five students for the number of 
students in this multigrade class. 


L|Class exclusion status (if applicable). As a rule, all classes are to be included. TIMSS 2015 will offer 
many accommodations that should allow most students to participate. Click on the Documents link on the 
left panel to see what accommodations are provided or allowed. All class-level exclusions must be 
approved. If you indicate a class-level exclusion, a TIMSS representative will contact you to discuss. 
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Examples of potential class-level exclusions include classes where all students belong to at least one of 
the following three exclusion status categories, and none of the students can be assessed with TIMSS 
accommodations: 


1 = students with functional disabilities. These are students who have physical disabilities in 
such a way that they cannot perform in the TIMSS testing situation. Students with functional 
disabilities who are able to perform should be accommodated in the test situation, within 
reason, rather than excluded. 


2 = students with intellectual disabilities. These are students who are considered, in the 
professional opinion of the school principal or by other qualified staff members, to have 
severe intellectual disabilities or who have been tested as such. This includes students who 
are emotionally or mentally unable to follow even the general instructions of the test. 
Students should not be excluded solely because of poor academic performance or normal 
disciplinary problems. It should be noted that students with dyslexia, or other such learning 
disabilities, should be accommodated in the test situation, within reason, rather than 
excluded. 


3 = non-native language speakers. These are students who are unable to read or speak the 
language(s) of the test and would be unable to overcome the language barrier in the test 
situation. 


If all students in the excluded class do not belong to the same exclusion category, please identify the 
category corresponding to the majority of students. 


|_|Name of mathematics teacher: Name of the teacher teaching mathematics content to the fourth-grade 
class. 


L_|Name of science teacher: Name of the teacher teaching science content to the fourth-grade class. 


TIMSS 2015 Grade 8: Class Roster Definitions 


The class roster definitions for grade 8 followed those described above for grade 4 except that science 
teacher was not collected in the class sampling form template. Rather, TIMSS also required the school to 
provide the list of eighth-grade science teachers in order to link information about science teachers to the 
assessed students. Schools were asked to enter each teacher’s name and the course the teacher taught 
(e.g., biology, integrated science) into the template (exhibit 4-4). 


Exhibit 4-4. Teacher list (grade 8 science teachers) 


x] | [Earth Science | (Jess Alban | 


[Biology 1 [Nate Halloran| 


Choose "Finished" once you have entered all mathematics 
- — classes and science courses for Grade 8. No further editing is 
Save | Print Finished possible when you choose this option. 


SOURCE: National Center for Education Statistics (NCES), Trends in International Mathematics and Science Study (TIMSS), 2015. 
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Sampling classes. For grades 4 and 8, WinW3S generated an equal probability sample of two classes (or 
pseudoclasses) where possible, based on the information in the CLF template. If only one grade- 
appropriate class was available in a school, this class was selected with certainty. 


4.2.2 Identifying Students and Their Teachers 


For TIMSS, schools were given the option of providing the list of the students in each of the mathematics 
classes listed in the Class Listing Form template or to wait for the results of the WinW3S class sampling 
selections. In the latter case, after class sampling was completed, Westat provided the schools with the 
sampled classes. The schools then provided Westat with the students of these selected classes using the 
Student Listing Form template. Westat provided the schools with multiple templates to allow them 
flexibility in completing the templates. The format of the student names and the format of the month and 
year of birth are often stored differently across schools so the templates attempt to simplify and lessen the 
burden on schools when they provide these data. Exhibit 4-5 presents the grade 4 student list template. 


Exhibit 4-5. Grade 4 student list template 


ClassName TeacherName Student Name Sex Date of Birth Student witha Disability Status English Language Learner Status 
SOURCE: National Center for Education Statistics (NCES), Trends in International Mathematics and Science Study (TIMSS), 2015. 


The grade 8 student list template requested more information than the grade 4 student list template as 
shown in exhibit 4-6. 


Exhibit 4-6. Grade 8 student template information requested 


Data element Description 
Mathematics Class The mathematics class the student is enrolled in. 


Please use the same class names you used in 
Submit Class List. 


Mathematics Teacher Teacher of the above named mathematics class 
that the student is taking 

Science Teacher Name of science teacher who teaches science to 
each student 

Student Name The preferred format is First Name, Middle Name 


(or Initial), and Last Name in separate columns. 
However, TIMSS will accept student names in 


one column. 
Sex Codes (numeric or text) for Male/Female 
Month and Year of Birth The preferred format is Month of Birth and Year 


of Birth in separate columns. However, TIMSS 
will accept both in one column. 


See note at end of exhibit. 
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Exhibit 4-6. Grade 8 student template information requested—Continued 


Data element Description 


Student with a Disability (SD) Status School-defined codes for: 
[_]Yes, IEP: Student has an Individualized 
Education Plan (IEP). Students with 504 plans 
that need accommodations can be coded this value 
as well. 
[]No, not SD: 
[_]Information unavailable at this time. 
English Language Learner (ELL) Status School-defined codes for: 
L]Yes, ELL 
L_]No, not ELL 
[_|Information unavailable at this time 


SOURCE: National Center for Education Statistics (NCES), Trends in International Mathematics and Science Study (TIMSS), 2015. 


For TIMSS Advanced, the courses that were identified as eligible for participation in the assessment 
based on the school course catalogs and interactions with the school were included in the templates sent 
to the schools. The schools listed all of the twelfth-grade students that had taken these courses in their 
junior and/or senior years of high school. The template to gather the student information used for the 
sampling assignments is shown in exhibit 4-7. 


For each eligible advanced course, schools were asked to provide the students and teachers of these 
classes, and furnish the information in this template for each respective subject: 


TIMSS Advanced requested a complete and current list of all twelfth-grade students who have taken or 
were currently taking eligible advanced mathematics and physics courses in order to draw a random 
sample of students to participate in the TIMSS Advanced assessment. More details of the student 
sampling procedures in TIMSS Advanced students are presented in chapter 2. Eligible courses for each 
school were sent to the school via email. The student data electronic file (E-File) must be submitted as a 
Microsoft Excel file. 


Exhibit 4-7. TIMSS Advanced student information template (advanced math) 


A B cc My «cl ri[ «fu 1 J kK 
Student ID provided by Student First Student Student Last Month Day of Year of Grade level when Advanced Advanced Mathematics Advanced Mathematics 
1 school Name Middle Initial Name of Birth Birth Birth Sex MathematicsCourse Taken Course Title Teacher Name 


SOURCE: National Center for Education Statistics (NCES), Trends in International Mathematics and Science Study (TIMSS) Advanced, 2015. 


A comparable template was provided to schools to collect TIMSS Advanced physics information. 


4.2.3 Student-Teacher Linkage Form and Student Tracking Form 


For TIMSS, once the class sampling has been conducted, the student lists submitted by the schools were 
imported into the WinW3S for each sampled class. Similar to TIMSS, for TIMSS Advanced, once the 
student sampling has been conducted from the student lists submitted by the participating schools, the list 
of sampled students was imported into the WinW3S. Students who were sampled for advanced 
mathematics were identified as session 01, and students were sampled for physics were identified as 
session 02 in the WinW3S software. The WinW3S then generated the Student-Teacher Linkage Forms 


U.S. TIMSS 2015 and TIMSS Advanced 1995 & 2015 Technical Report and User’s Guide 4-11 


(STLF) and Student Tracking Forms (STF) for each sampled class for TIMSS grades 4 and 8, and for 
each sampled session for TIMSS Advanced. The STLFs and STFS were provided to the schools around 
the assessment time. These forms were transmitted back to the schools for further updates. The Student 
Tracking Form was designed to provide test administrators with student IDs, student identifiers in the 
form of date of birth and sex, the booklet assignment to each student, and the means to record the 
completion of the assessment and associated questionnaire. The student names shown in the first column 
of the form were removed following the assessment and retained by the school to ensure confidentiality. 
An example of an STF containing fictitious information is provided in exhibit 4-9. The participation and 
eligibility status were recorded in the electronic form and imported into the WinW3S to update the 
participation fields. 


The primary purpose of the STLF is to link students to their mathematics and/or science teachers and 
classes. School staff and field staff ensured that the data found on both of the forms were correct and 
matched the student records from the school. Discrepancies were reviewed, and edits were made to the 
forms. Assessment booklets were assigned based on the booklet assignment from the WinW3S generation 
of the STF. Examples of an STLF and an SLF containing fictitious information are provided in exhibits 
4-8 and 4-9, respectively, for TIMSS grade 8. Because the STF and STLF provide the same level of 
information for TIMSS grade 4, grade 8, and TIMSS Advanced, the examples shown are sufficient for the 
understanding of their function. The STLF forms were returned to Westat where the links were entered 
into the WinW33S as the software does not yet allow the automation for this function. The data found on 
the STF, however, were automated, and all updates to student status and participation on the form were 
imported into the WinW3S. 


As the school and teacher questionnaires were conducted online in the IEA Online Survey System (OSS), 


the tracking of teacher and school participation in the survey could be automated. This system contained a 
monitoring feature to track completion of teacher and school surveys. 
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Exhibit 4-8. Example of TIMSS eighth-grade Student-Teacher Linkage Form (STLF) with mock data 


TIMSS 2015 - 
Student-Teacher Linkage Form - Grade 8 


School Name: TIMSS Participant Country: 
USA 
Smarty School [al [b] [c] [d] 


TIMSS_ID_1666666 


ID Class ID Class Name Grade 
9991 999103 Period 5 8 


eS) 


oe @ 3) 


Teacher Information 


Gayle Legenza 
Cilla Olsson 


Student Name 
or Number 


Damen, David 
Bateman, Eric 
Lew, Moise 
Darienza, Toby 
Schroeder, Joanne 
Kaplan, Stuart 
Rosen, Susan 


Exclusion/ Accommodation Status 


Sequence Number 
Date of Birth 
Teacher’s Name: 
Class (Course) Name: 
Subject Code: 
Teacher’s Name: 
Subject Code: 
Teacher’s Name: 
Class (Course) Name: 
Subject Code: 
Teacher’s Name: 
Class (Course) Name: 
Subject Code: 
Teacher’s Name: 
Class (Course) Name: 
Subject Code: 
Teacher’s Name: 
Class (Course) Name: 
Subject Code: 
We Wee Wee ae ae cl No Mathematics Teacher 
No Science Teacher 


DD | MM | YYYY 


pate Class (Course) Name: 


Gender (column 4): 1 = Female; 2 = Male 

Exclusion/Accommodation Status (column 5): 1 = Students with functional disabilities; 

2 = Students with intellectual disabilities; 3 = Non-native language speakers; 4 = Requires Accommodations 

Subject Codes (Column 6): 1 = Mathematics; 2 = Physics; 3 = Biology; 4 = Chemistry; 

5 = Earth Science; 6 = Integrated Science; 8 = Physics/Chemistry; 9 = Biology/Earth Science; 

10 = Biology/Chemistry; 11 = Physics/Biology; 12 = Physics/Earth Science; 

13 = Chemistry/Earth Science; 14 = Natural Science; 15 = Scientific Work; 16 = <Country Specific 1>; 

17 = <Country Specific 2> 
SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS), 2015. 
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Exhibit 4-9. Example of TIMSS eighth-grade Student Tracking Form (STF) with mock data 


TIMSS 2015 - Student Tracking Form - Grade 8 


School Name: TI MSS Participant Country: 
USA 
La] [b] [c] Ld] Le] 


Smarty School Class Language 
TIMSS_ID_1666666 School ID Class ID Name of Test 


Olsson 
999103 Period 5 


6 8 @®@ 7] 


i) 
iN) 


Date of Birth ® 


Participation Status 


Main 
Session Make-Up Session 


Student Name Student 
or Number ID Dp |mm |yyyy 


Damen, David * 99910301 
Bateman, Eric + 99910302 
Lew, Moise ' 99910303 


Exclusion/ Accommodation 


Status 
Questionnaire Session 


Achievement Session 
Questionnaire Session 
Achievement Session 


Booklet 


co} Ww 
Pel Ll Di Gender 
ou Ayu 


Darienza, Toby * 99910304 2 
Schroeder, Joanne » 99910305 2001 
Kaplan, Stuart : 99910306 2001 
Rosen, Susan ' 99910307 2001 
<A> ” 99910308 
he ” 99910309 
12 


<A> ' 10610310 


Ol! 
an) 


PyR SO ~N 
Plo 
wn 


Gender (column 4): 1 = Female; 2 = Male 

Exclusion/Accommodation Status (column 5): 1 = Students with functional disabilities; 

2 = Students with intellectual disabilities; 3 = Non-native language speakers; 4 = Requires accommodations 
Participation Status (column 7): C = Participated; SA = Participated with special accommodation; 

A = Absent; P = No parental permission; NA = Left school permanently 


SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS), 2015. 
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4.3 Instruments for TIMSS and TIMSS Advanced 


All TIMSS and TIMSS Advanced instruments were developed by the IEA as a collaborative effort 
involving representatives from every country participating in the study. For TIMSS, at each grade the 
primary instruments for TIMSS 2015 consisted of a combined mathematics and science assessment; a 
student questionnaire; a curriculum questionnaire; a school questionnaire; and teacher questionnaires to 
be completed by the teachers teaching mathematics and/or science to the students in the sampled 
classrooms. For TIMSS Advanced the instruments included a physics assessment; an advanced 
mathematics assessment; a physics student questionnaire; an advanced mathematics student 
questionnaire; a curriculum questionnaire; a school questionnaire; and teacher questionnaires to be 
completed by the teachers teaching physics or advanced mathematics to the sampled students. 


The TIMSS assessment instrument took the form of 14 booklets containing mathematics and science 
items in rotated item blocks. While some of these items have to remain confidential for use in coming 
TIMSS assessments, others are available on a restricted use basis. An IEA permission request form to 
access and use the TIMSS 2015 restricted items is available at 
http://timssandpirls.bc.edu/timss2015/international-database/assessment-items. html. 


The assessment for TIMSS Advanced consisted of 6 booklets for physics and 6 booklets for advanced 
mathematics and presented items in rotated item blocks. Like TIMSS, some items have to remain 
confidential for use in coming TIMSS Advanced assessments while others are available on a restricted- 
use basis. The permission request form for TIMSS Advanced restricted use items is available at 
http://timssandpirls.bc.edu/timss2015/advanced-international-database/. 


The TIMSS student questionnaires were grade-specific and bound into the assessment booklet with the 
assessment items for the particular grade. TIMSS Advanced student questionnaires were also bound into 
either the physics or advanced mathematics assessment booklet. The school questionnaires were designed 
as an online instrument to be completed by the school principal for both TIMSS and TIMSS Advanced. 
Teacher questionnaires were also developed as online instruments. A single fourth-grade online teacher 
questionnaire was designed for TIMSS mathematics and/or science teachers of the students in the 
sampled classrooms. Separate questionnaires were provided for the TIMSS mathematics teachers and 
science teachers of the students in the sampled eighth-grade classrooms. TIMSS Advanced physics and 
advanced mathematics teacher questionnaires were provided for teachers of students taking either a 
physics or advanced mathematics assessment. 


The international versions of the TIMSS questionnaires are available on the TIMSS & PIRLS 
International Study Center website at http://timssandpirls.bc.edu/timss2015/questionnaires/index.html. 
The international versions of the TIMSS Advanced questionnaires are available at 
http://timssandpirls.bc.edu/timss2015/advanced-questionnaires/index.html. The U.S. versions of all seven 
U.S. TIMSS questionnaires and the five TIMSS Advanced questionnaires are provided in appendix D of 
this report and are available on the NCES website http://nces.ed.gov/timss/questionnaire.asp. 


4.3.1 The TIMSS Mathematics and Science Assessment and TIMSS Advanced 
Physics and Advanced Mathematics Assessment 


The following summarizes the rationale for and development of the TIMSS and TIMSS Advanced 2015 
assessments. Complete detail is provided in the Methods and Procedures in TIMSS 2015 (Martin, Mullis, 
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and Hooper 2016a) and Methods and Procedures in TIMSS Advanced 2015 (Martin, Mullis, and Hooper 
2016b). 


Assessment Frameworks 


For both TIMSS and TIMSS Advanced, the test development effort began with a revision of the 
frameworks that define the knowledge and skills assessed in previous TIMSS 2011 and TIMSS Advanced 
2008 assessments (Mullis et al. 2009; Garden et al. 2006). The frameworks were updated to reflect 
changes in the curriculum and instruction of participating countries. Extensive input from U.S. and 
international experts in mathematics, science, and measurement contributed to the final shape of the 
frameworks. Maintaining the ability to measure change over time was an important factor in revising the 
frameworks. The complete subject area frameworks for TIMSS and TIMSS Advanced 2015 are available 
at http://timssandpirls.bc.edu/timss2015/frameworks.html and http://timssandpirls.bc.edu/timss2015- 
advanced/frameworks.html. 


Content and Cognitive Domains 


The TIMSS and TIMSS Advanced assessments measure students’ knowledge and skills across specific 
content domains defined for each grade and subject (exhibit 4-10). The assessment items across these 
content domains measure what students can do across a range of cognitive skills or processes: knowing, 
applying, and reasoning. TIMSS and TIMSS Advanced science and physics frameworks also describe the 
science inquiry practices to be measured. 


Exhibit 4-10. Content domains in TIMSS and TIMSS Advanced 2015 


TIMSS assessment Mathematics Science 

TIMSS 2015 Grade 4 Number Life science 
Geometric shapes and measures _ Physical science 
Data display Earth science 

TIMSS 2015 Grade 8 Number Biology 
Algebra Chemistry 
Geometry Physics 
Data and chance Earth science 

TIMSS Advanced 2015 

(Grade 12) (Advanced mathematics) (Physics) 

Algebra Mechanics and thermodynamics 
Calculus Electricity and magnetism 
Geometry Wave phenomena and atomic/ 


nuclear physics 


SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS) 2015; Trends in International Mathematics and Science Study (TIMSS) Advanced 2015. 
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Item Development 


Over one-half of the assessment items used in TIMSS 2011 and in TIMSS Advanced 2008 were kept 
confidential and included in the respective TIMSS and TIMSS Advanced 2015 assessments to measure 
trends. To replace assessment items that had been released following the previous assessments, education 
systems submitted items for review by subject-matter specialists. Additional items were written by the 
International Association for the Evaluation of Educational Achievement (IEA) Science and Mathematics 
Review Committees (SMIRC) for TIMSS and TIMSS Advanced and by item-writing and subject area 
specialists. The items were reviewed by participating countries, the SMIRC, and subject matter experts in 
assessment to ensure that the content, as defined in the frameworks, was covered adequately. Items were 
field-tested in most of the participating education systems. Results from the field test were used to 
evaluate item difficulty, how well items discriminated between high- and low-performing students, the 
effectiveness of distracters in multiple-choice items, scoring suitability and reliability for constructed- 
response items, and evidence of bias toward or against individual countries or in favor of boys or girls. In 
addition, cognitive labs were conducted in the United States for 20 grade 4 mathematics, 20 grade 4 
science, 20 grade 8 mathematics, 20 grade 8 mathematics, 20 advanced mathematics and 36 physics items 
to evaluate the clarity, format, and content of these items. 


The number of new and trend mathematics and science items for the fourth- and eighth-grade TIMSS and 
the TIMSS Advanced assessment are shown in table 4-1. Across both grades 4 and 8 the assessments 
contained 777 items, of which 458 were trend items and 319 were new items. The number of items in 
TIMSS Advanced was increased by about 30 percent from the 2008 assessment to provide better 
coverage of the advanced mathematics and physics content and cognitive domains. The entire TIMSS 
Advanced assessment included 205 items, of which 63 were trend items and 142 were new items. More 
information about the TIMSS and TIMSS Advanced 2015 item development can be found in Chapter 1 of 
Methods and Procedures in TIMSS 2015 (Martin, Mullis, and Hooper 2016a) and Methods and 
Procedures in TIMSS Advanced 2015 (Martin, Mullis, and Hooper 2016b). 


Table 4-1. Number of trend and new assessment items by grade and subject, TIMSS 2015 and TIMSS 


Advanced 2015 
TIMSS assessment Mathematics Science Total 


TIMSS 2015 Grade 4 


Total 169 176 345 
Trend 102 101 203 
New 67 75 142 


TIMSS 2015 Grade 8 


Total 212 220 432 
Trend 127 128 319 
New 85 92 177 
TIMSS Advanced (Grade 12) (Advanced mathematics) (Physics) Total 

Total 102 103 205 
Trend 32 31 63 
New 70 72 142 


SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Trends in International Mathematics and Science 
Study (TIMSS), 2015; Trends in International Mathematics and Science Study (TIMSS) Advanced, 2015. (Add if table approved.) 
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Assessment Booklets 


To keep the testing burden to a minimum while ensuring broad subject matter coverage, TIMSS used a 
rotated block design that included both mathematics and science items. This is consistent with other large- 
scale assessments such as the National Assessment of Educational Progress (NAEP). The rotated item 
design meant that, while no students responded to all of the items, each student encountered both 
mathematics and science items during the assessment. 


The 2015 fourth- and eighth-grade assessments consisted of 14 booklets, assembled from 28 distinct item 
blocks at each grade level. Fourth-grade blocks contained approximately 10—14 assessment items, while 
eighth-grade blocks contained 12—18 items. Each booklet contained four blocks, including at least one 
mathematics and at least one science block. The fourth-grade booklets required about 72 minutes, and 
eighth-grade booklets required about 90 minutes. 


The TIMSS Advanced assessments consisted of 6 booklets for advanced mathematics and 6 booklets for 
physics, assembled from 9 distinct blocks, each requiring approximately 90 minutes. Each student 
completed just one booklet. In both subjects, booklets consisted of 3 blocks each. 


Calculator Use in TIMSS and TIMSS Advanced 


Calculator use was not permitted during the TIMSS fourth-grade assessment. However, the TIMSS policy 
on calculator use at eighth grade was to give students the best opportunity to operate in settings that 
mirrored their classroom experiences. As a consequence, calculators were permitted, but not required, for 
the eighth-grade assessment materials. In the United States this meant that students in the eighth-grade 
were allowed, but not required, to use calculators. 


The TIMSS Advanced policy allowed the use of calculators on both the physics and advanced math 
assessments. Students were instructed to bring their personal calculators to the assessment as long as they 
were not devices with access to the Internet. The calculators could not be a laptop or other portable 
computer, pocket organizer, cell phone, device with a typewriter-style keyboard, electronic writing pad, 
or pen-input device. 


During the assessment the TIMSS Advanced test administrator confirmed the calculators met the 
requirements and monitored their use by the students. 


Context Questionnaires 


As in prior administrations, TIMSS and TIMSS Advanced 2015 included self-administered 
questionnaires for principals, teachers, and students. To create the questionnaires for 2015, the 2011 
versions of the TIMSS questionnaires and the 2008 versions of the TIMSS Advanced questionnaires were 
reviewed extensively by the National Research Coordinators from the participating countries as well as 
the Questionnaire Item Review Committee (QIRC). Based on this review, the QIRC eliminated or revised 
some questions and added several new ones. Like the assessment items, all questionnaire items were 
field-tested and the results reviewed. As a consequence, some of the questionnaire items were revised 
prior to their inclusion in the final questionnaires. 
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The questionnaires requested information to help provide a context for the performance scores, focusing 
on such topics as students’ attitudes and beliefs about learning, their habits and homework, and their lives 
both in and outside of school; teachers’ attitudes and beliefs about teaching and learning, teaching 
assignments, class size and organization, instructional practices, and participation in professional 
development activities; and principals’ viewpoints on policy and budget responsibilities, curriculum and 
instruction issues, and student behavior, as well as descriptions of the organization of schools and 
courses. For 2015, online versions of the school and teacher questionnaires were offered to respondents as 
the primary mode of data collection. Paper versions were also available, if requested. 


U.S. Adaptations to the Assessment Items and Questionnaires 


Source versions of all instruments (assessment booklets, questionnaires, and manuals) were prepared by 
the IEA in English and translated by the participating countries into the primary language or languages of 
instruction in each country. In addition, it was sometimes necessary to adapt the instruments to better fit 
language usage, even in countries that use English as the primary language of instruction. Other 
adaptations to fit national education characteristics were sometimes required as well. All adaptations were 
reviewed and approved by the IEA to ensure they did not change the substance or intent of the question or 
answer choices. 


U.S. Adaptations to the Assessment Items 


As in previous cycles of TIMSS and TIMSS Advanced, the U.S. adaptations to the international 
instruments were minimal and designed to make the assessment more readable to U.S. students without 
changing the essence of the assessment item. For example, at times names of individuals were changed to 
more familiar forms (for example, “Ahmed” to “Andrew”), nouns with British origins were changed to 
their U.S. equivalent (for example, “cinema” to “movie theater”), Imperial English spellings were 
changed to American English (for example, “organisation” to “organization,” “programme” to 
“program”), and some changes to the text of instructions were also made to better mirror the 
administration procedures in the United States. 


U.S. Adaptations to the School, Teacher, and Student Questionnaires 


Adaptations made to the school, teacher, and student questionnaires were of the following five main 
types: 


changes to general instructions made in the interests of enhancing clarity; 


7 changes designed to make question text more readable to U.S. students, similar to 
those made to the assessment items as described above; 


7 changes to response alternatives where the international response set did not 
adequately reflect the U.S. context; 
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a additional questionnaire items included to address particular issues of national 
interest; and 


Ly changes to ensure some questions were kept in trend with previous assessments. 


A detailed list of changes made to the questionnaires is provided in appendix E. Both the original text 
from the international version of the questionnaire and the changed text from the U.S. version are shown. 
Text that has been changed in the U.S. version is underlined in that version. Both international and U.S. 
questionnaire item numbers, or other location indicators, are provided in each instance. Where 
appropriate, a crosswalk between the U.S. and international versions of the set of response categories of 
items is provided in the “Comments” column. 


Translation and Verification of Instruments 


Each country prepared translations of the instruments according to translation guidelines established by 
the IEA. As the international versions of the instruments are produced in English, the U.S. did not need to 
engage in the full-fledged translation required of many nations. However, the adaptations made to the 
U.S. instruments required verification by the IEA to ensure their suitability for the current cycle of 
TIMSS and, if trend items, their continuity with previous cycles. Further details on the translation process 
can be found in Martin, Mullis, and Hooper 2016a and Martin, Mullis, and Hooper 2016b. 


Production of Assessment Booklets and Questionnaires 


Upon receiving IEA approval of the adaptations, Pearson applied the adaptations to the international 
questionnaires and item blocks and then assembled the final assessment booklets. Quality control 
procedures for this process included a review of each adaptation made to the questionnaires and item 
blocks as well as a full review of the assembled instruments in a final layout proof. 


In mid-December 2014, electronic files were sent to the TIMSS & PIRLS International Study Center for 
layout verifications. Final approval to print was given in early February 2015. The student assessment 
booklets and student questionnaires were printed on scannable form. The teacher and school paper and 
pencil versions of the questionnaires were printed on non-scannable form. 


A similar procedure was applied to the online school and teacher questionnaires. Adaptations were made 
to the online instruments using software provided by the IEA Data Processing and Research Center 
(DPC). A set of output files was produced for each questionnaire, and these were uploaded to a secure 
server and verified by the IEA DPC. 


Scannable Document Preparation and Printing 


The student cognitive assessments and student questionnaires were printed as scannable documents 
combined into 14 single booklets at grades 4 and 8 for TIMSS, and six physics and six advanced 
mathematics booklets for TIMSS Advanced. Document production was divided into two phases. The 
preparation phase included the mockup and design of forms, typesetting, composition, and editing for text 
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accuracy and process ability. The production phase included final platemaking, printing, binding, and any 
finishing procedures (counting, wrapping, etc.) that were required prior to packaging and distribution. 


After a form was created, it was thoroughly inspected for grammar, spelling, and punctuation to ensure 
that it matched the approved electronic files. Subsequently, copies of these proofs were subject to a 
technical review of “scan ability,” oval placement, and spine code assignment. Immediately after printing, 
sample documents were selected from predetermined locations throughout the print run for testing. Before 
shipping, a sample from each carton of multipage documents was inspected to ensure that the pages of the 
booklets were in the correct sequence. After binding, all documents were boxed to ensure that material 
quality was maintained during transit to the packaging facility. 


Preparation and Printing of Non-Scannable Documents 


The teacher and school questionnaires were produced as non-scannable documents. In the first stage, 
proofs of each document were reviewed against the original electronic files. Once accuracy was certified, 
printing was initiated. During this process staff checked a 10 percent sample of the printed form against 
the approved document to ensure that accuracy was maintained throughout the printing process. 


Online Questionnaires 


Once verified by IEA DPC, each questionnaire for school and teachers was loaded to a secure NCES 
server and thoroughly tested by NCES and Westat to ensure that responses were being captured correctly 
and that the instruments were functioning properly. 


4.4 Field Operations 


The activities discussed here refer to those associated with the administration of the assessments in 
participating schools. In the United States the administration of the assessments was carried out by 
professional staff trained according to the international guidelines. School personnel were asked only to 
assist with listings of students, the identification of school space for the assessment, and the specification 
of parental consent procedures needed for sampled students. 


Field operations centered on three main tasks: recruiting and training of field staff, scheduling the 
assessments, and administration of the assessments within the schools. 


Field staff consisted of test administrators and assistant administrators. Test administrators managed the 
assessments in the schools, while assistant administrators assisted in the assessment administration, 
conducting one of the assessments because in most schools two separate classrooms were assessed 
simultaneously. All had previous experience with other educational assessments in schools, and all had 
FBI clearance based on fingerprint and background checks, as well as eQIP security clearances. Test 
administrators and assistant administrators also signed a statement of nondisclosure indicating that they 
would maintain confidentiality of all survey materials and of the data collected. 


The total complement of field staff consisted of one field director, 5 field managers, 64 test 
administrators, and 64 assistant administrators. The field director reported directly to the Westat home 
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office and met weekly to discuss progress and any problematic issues arising in the field. Field managers 
reported to the field director, Test administrators reported to a field manager who coordinated and 
monitored their work and, in turn, the test administrators coordinated and supervised the work of the 
assistant administrators. 


4.4.1 Responsibilities of Field Director, Field Managers, Test Administrators 
and Assistant Administrators 


Field director and field managers involved in the data collection phase had responsibility for: 


providing input to the TIMSS and TIMSS Advanced data collection in-person 
training; 


tracking the test administrators’ receipt of assessment booklets and other materials; 
coordinating data collection activities undertaken by their assigned test administrators; 


holding weekly one-on-one telephone meetings with their test administrators to 
monitor progress and to troubleshoot any problems arising; 


ensuring that their test administrators followed TIMSS and TIMSS Advanced 
procedures and guidelines; and 


reporting progress and problems in weekly conference calls with Westat home office 
staff and other field managers. 


Test administrators had responsibility for: 


attending TIMSS and TIMSS Advanced data collection in-person training; 
receiving and securing assessment materials; 
training assistant administrators; 


working with the school coordinator to notify parents and students of the assessment, 
and obtain permission of parents, if this was required by the school; 


making arrangements with the school for the assessment sessions; 
preparing and assigning assessment materials for students; 


monitoring online completion of the school and teacher questionnaires and collecting 
questionnaires completed in hard copy, as applicable; 


conducting the assessment according to TIMSS and TIMSS Advanced specified 
procedures; 


completing the Test Administration Form, Student Tracking Form, Teacher Tracking 
Form, and Student Response Rate Form; 
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5 determining if a makeup session was needed; 


a securing, packing, and shipping all assessment materials to Pearson at the conclusion 
of the assessment for scoring; 


a recording the status of the assessment in the MyTIMSS School Control System; and 
ey reporting progress to their field manager on a regular basis. 
Assistant administrators had responsibility for 


7 attending the TIMSS and TIMSS Advanced in-person training conducted by their test 
administrator; 


7 administering the assessment according to TIMSS and TIMSS Advanced specified 
procedures; 


rT completing the Test Administration Form, Student Tracking Form, Teacher Tracking 
Form, and Student Response Rate Form; and 


7 consulting regularly with their test administrator. 


4.4.2 Training 


A 2'%-day, in-person training for test administrators was held on March 11-13, 2015. The attendees 
received the Test Administrator Manual 5 days prior to the training session and were given four paid 
“study hours” to become familiar with the information prior to training. The agenda for this training 
session is provided as exhibit C-1 in appendix C. 


TIMSS and TIMSS Advanced were very similar in terms of general procedures; therefore the training 
was centered on TIMSS. TIMSS Advanced was referenced at specific times when a unique training point 
needed to be made or when procedures differed. The first day of training focused on the responsibilities of 
a TIMSS test administrator, TIMSS data collection materials, the Preassessment Call Checklist, and 
entering preassessment call information into the MyTIMSS School Control System. In addition, 
accommodations allowed on TIMSS and increasing grade 12 student participation were discussed. The 
second day included discussion about the procedures to be followed in preparing for, arriving on, and 
conducting the assessment day. The third half day dealt with what to do once the assessment was 
completed, and the appropriate methods for packing and shipping the assessment materials to Pearson 
Educational Measurement. Privacy and security of test materials was reviewed and test administrators met 
with their field managers to review assignments and schedules. 


Field staff were assigned laptop computers to take with them for the duration of TIMSS and TIMSS 
Advanced data collection. Printers for use with the laptops were issued and sent to each test administrator 
after training. Test administrators were also provided with an official TIMSS/TIMSS Advanced 2015 
photo ID badge to wear while representing TIMSS and TIMSS Advanced in the schools. 


Training for assistant administrators was conducted as a separate exercise by their respective test 
administrator. Approximately 4 hours was allocated for this training. Training materials consisted of an 
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Assistant Administrator Manual, session scripts, and a subset of material from the test administrator 
training on assessment day tasks. 


4.4.3 Scheduling Assessments 


The approximately 900 schools taking part in TIMSS and TIMSS Advanced 2015 were dispersed across 
the country, although with concentrations in the more populous states and cities. Scheduling assessment 
dates for these schools required the optimization of school preferences for a particular date with the 
assessment date preferences of nearby schools and the location of field staff in an effort to keep travel and 
related expenses to a minimum. The basic approach adopted involved the geographic mapping of schools 
and assignment of a preliminary date, along with the location of field staff. This formed the foundation 
for discussions with schools, and the assignment of schools to field staff. In essence, geographical clusters 
of schools were assigned assessment dates clustered in time insofar as this was possible. 


Obtaining Assessment Dates From Schools 


Prior to recruitment of schools, Westat staff collected date information for the sampled schools, including 
dates associated with state testing, holidays, breaks, field trips, and the like. This information was used to 
set a tentative date for each school. 


Mapping Schools 


In a second step a geographic map of participating grade 4, grade 8, and grade 12 schools was developed 
based on the addresses of the schools. The map allowed the ready identification of obvious clusters of 
schools in large metropolitan areas and also allowed for identification of less concentrated clusters in 
broader geographical areas as well as isolated schools. To establish optimal regions for assessment 
administrators, geographic clusters of schools were defined in an iterative process that took into account 
the location of field staff and their caseload. A total of 59 regions of varying size were identified in this 
way. 


Assigning Tentative Assessment Dates 


The provisional assessment dates were then mapped to the schools in each cluster and represented in the 
MyTIMSS School Control System. Within clusters, assessment dates were balanced against the location 
of schools in relation to one another. With some minor modifications to regions, schools in relatively 
close proximity were assigned to provisional assessment weeks. This process identified smaller clusters 
of schools that could be assessed within a week without undue travel and related expenses. For the most 
part, assessments were scheduled on Tuesday through Thursday, leaving Monday and Friday as travel 
days. 
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Negotiating Final Assessment Dates 


Once tentative dates were established, field staff contacted schools to negotiate a final assessment date. 
For the most part, schools accepted the dates proposed. When this was not possible, field staff used the 
work area spreadsheet and a work area map to negotiate an alternative assessment date that would 
minimize staff travel. Final assessment dates and times were confirmed by email with the school 
coordinator. 


4.4.4 Assignment of Schools to Test Administrators 


For the most part, field staff were assigned a region based on their location and availability with most 
regions relatively close to the test administrator’s home address. Balancing these several demands 
resulted in some variation in the caseload of test administrators. During the course of the assessments, 
which ran from end of March 2015 through to early June 2015, some reassignments of schools and/or 
work areas were necessary. 


Troubleshooting 


Five of the test administrators and five assistant administrators were recruited as regional troubleshooters 
who would be available to cover last-minute changes in assessment dates, staff illnesses, and the like. 
Each of these staff also had their own assignments, essentially schools that did not easily fit into an 
identified region or assessment date window. 


4.4.5 Recruiting Parents and Students 


During recruitment and scheduling contacts with schools, field staff asked about district and/or school 
requirements for notifying parents about their child’s participation in TIMSS or TIMSS Advanced. 
School requirements fell into the following three main categories: 


Ey Notification. The school would simply send parents a notification of the child’s 
participation in TIMSS along with some informational material; 


5 Passive consent. The school was required to ask parents for permission for the child 
to participate but permission would be assumed unless there was a formal objection; 
and 

7 Active consent. The school was required to ask parents for permission for the child to 
participate and the child could not participate until the parents provided formal 
approval. 


Almost all schools opted for parental notification. 


To assist schools in this task the school coordinator was provided with one of three draft letters to parents, 
one for each of the three forms of parent permission. These letters could be edited as appropriate and sent 
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out on school stationery. Consent forms to accompany the passive and active consent letters were also 
provided along with an information sheet describing TIMSS or TIMSS Advanced. English and Spanish 
versions of each of these documents were made available to the schools. Copies of these materials as they 
apply to fourth-grade, eighth-grade and Advanced students are provided as exhibits A-5 through A-20 in 
appendix A. 


4.4.6 Organizing the Assessment Session at the School 


Approximately four to two weeks prior to each school’s assessment date, the test administrator called the 
assigned school coordinator. Using a Preassessment Call Log, test administrators were instructed to verify 
previously obtained information on items such as the school’s address, principal’s name, assessment date, 
session location, requirements for entering the school, and parking arrangements as well as the status of 
the within-school sampling forms for the school. The information obtained was maintained within the 
School Folder and updated as a basic reference. The Preassessment Call Log is provided as exhibit C-2 in 
appendix C. 


On assessment day each test administrator, accompanied by an assistant administrator, arrived at the 
school with all of the materials needed for the assessment. One session box of materials was provided for 
each of the sampled classes. For TIMSS Advanced, physics and advanced mathematics materials were in 
separate session boxes, for sessions of students selected for either physics or advanced mathematics. Each 
session box contained the estimated number of student assessment booklets required, plus three 
unassigned booklets to accommodate any changes in class enrollments at grades 4 and 8. Upon arrival, 
the test administrator met with the school coordinator to make any updates to the Student Tracking Form 
that would affect the preparation of student materials (for example, the addition of new students, the 
withdrawal of listed students from the school or class, or a change in exclusion status of a sampled 
student). 


Test administrators arrived at the school about an hour before the scheduled assessment to prepare 
booklets for distribution and take care of other arrangements for the assessment. Following the prescribed 
international procedures, the test administrator did not open the booklet bundles until 45 minutes before 
the assessment. At that time, the booklets were assigned to students in the random order established by 
the IEA sampling software, and labels were placed on the booklets. TIMSS pencils were provided to all 
students. As required by international rules, simple-function calculators were provided at the eighth-grade 
level only, and then only at the discretion of the school. Students participating in the TIMSS Advanced 
assessment were instructed to bring their own scientific calculator to the assessment. Students kept the 
TIMSS pencils as gifts, but calculators distributed during the assessments were collected with the 
booklets after the assessment. 


Administering the Assessment 


Assessments were administered by reading verbatim from a standardized script according to the 
instructions in the TIMSS and TIMSS Advanced Test Administrator Manuals. A copy of each of the 
sessions scripts are provided in appendix C. The script began with a brief introduction to the study. The 
assessment booklets, each in a security envelope, then were distributed. The students were instructed to 
remove their booklet from the envelope, and the general instructions and instructions for Part 1 were read. 
Following this, the students were instructed to begin Part 1 of the assessment. After the allotted 36/45 
minutes (grade 4/grade 8/12), a short break was provided. After the break, the instructions for Part 2 were 
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read and students were instructed to begin Part 2 of the assessment. After the allotted 36/45 minutes for 
this part of the assessment, students were instructed to stop work, and a longer break was provided. 
Following the break, the student questionnaire was administered; it was not time-limited but was typically 
completed in about 30 minutes. While students completed each section of the assessment, the test 
administrator checked that the students were working in the correct section of the assessment booklet, 
answered any questions students had, and ensured the allotted section timing was followed as specified by 
the assessment procedures and guidelines. 


Postassessment Activities 


Following the assessment, test administrators instructed the students to remove an identifying label from 
the cover of the booklet, place the booklets in the security envelope, and seal it. The students handed their 
booklets to the administrator, received their gift, and were dismissed. The test administrator then recorded 
participation codes for each session and packed the booklets and school and teacher questionnaires into 
the shipping box. The test administrators made copies of the Student Tracking Form and the Session 
Report Form, placing the original documents in the TIMSS and/or TIMSS Advanced Storage Envelope to 
be kept at the school. To maintain the security of student names, the test administrator removed and 
destroyed the column of student names from the copies of the STF and placed the de-identified STF and 
Session Report Form in with the assessment booklets and questionnaires. The session materials were 
sealed and shipped to Pearson. 


4.5 Receipt Control, Scoring, Data Entry, and Editing 


Westat field staff sent the completed assessment materials along with any related materials directly to 
Pearson following the completion of the assessment session at a school. Pearson recorded the receipt of 
materials, scored the open-ended responses, captured and edited the multiple-choice assessment items, 
and created data files from this information. The data files were sent to Westat. 


4.5.1 Receipt Control 


TIMSS documents were received at Pearson from April 1, 2015 through June 9, 2015. As assessment 
materials were returned to Pearson, receipt dates were recorded and sent to Westat twice a week. Pearson 
updated the electronic student tracking forms with the participation status and any other updates made by 
the test administrators during and after testing. These electronic files were sent to Westat on a weekly 
basis. 


Pearson checked in and processed assessment booklets from approximately 312 grade 4, 311 grade 8 and 
271 grade 12 schools. This included the additional schools from Florida, which participated as a 
Benchmarking state. School personnel had the option to complete their teacher and school questionnaires 
online or complete a hard copy and return them with the assessment booklets. Those returned to Pearson 
were sent to the Westat home office for entry into the online system. 
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Receipt Control System 


Two systems were used by Pearson to monitor the receipt and processing of assessment booklets after the 
assessments—the Process Control System and Workflow Management system. These systems enabled 
Pearson’s project staff to determine the status of any selected school; verify materials from a completed 
school had been received; identify discrepancies in student or school information; and obtain information 
on the status of data processing activities for a particular batch of booklets. 


The Process Control System contained a list of all participating schools and monitored the status of all 
sampled schools and assessment booklets. School materials were returned to Pearson packaged in their 
original boxes. A barcoded label containing the school ID was applied prior to shipping the booklets out 
to the Westat administrators. This barcode was scanned upon arrival at Pearson. Daily receipt reports 
were provided to project staff. 


Opening staff checked each school shipment to verify that the contents in the box matched the school ID 
indicated on the label. Each school shipment was then checked for completeness and accuracy based on 
procedures outlined in the IEA Survey Operations Manual. Any discrepancies were recorded and project 
staff alerted to determine the cause. Once a school shipment was opened and verified as being complete, 
the booklets were organized into work units and batched. The computerized Workflow Management 
System allows the user to track the flow of work through every processing step. 


Booklet Accountability 


Prior to the distribution of materials, a barcode was applied to the assessment booklets. These booklets 
were then organized into bundles. The barcodes within each bundle were scanned to a file that was used 
to control distribution to field staff or a particular school. This assignment of bundles to schools and/or 
administrators was recorded in the Materials Distribution System. 


After receipt of the school shipment after the assessment, a manual count was made to ensure all booklets 
from the original bundle were present. The assessment booklets were submitted for scanning and editing. 
The unused booklets were batched and the booklet identification barcode scanned. This file and the 
processed documents file were compared to the original bundle security file created before distribution. A 
report listing the unmatched booklet identification numbers was printed and used to confirm any 
nonreceipt of individual booklets. This report was given to the Westat home office. All assessment 
booklets were returned to Pearson. 


After the batches of documents had successfully passed the scanning and editing process, they were sent 
to the warehouse for storage. The storage locations of all documents were recorded in Pearson’s inventory 


control system, which permits the rapid retrieval of any document, should it be necessary. Used and 
unused materials are securely stored until permission is given to destroy them. 


4.5.2 Scoring the Assessment Items 


In the United States, the scoring of the TIMSS and TIMSS Advanced open-ended student responses was 
the responsibility of Pearson. Scoring rubrics developed internationally following the field test of the 
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assessment items were available to guide the scoring of each open-ended constructed-response item in the 
Main study. 


Training 


The United States sent one scoring representative to attend the grade 4 and grade 8 international scoring 
training in Willington, New Zealand in November 2014. Two additional scoring representatives were sent 
to the TIMSS Advanced international scoring training in Prague, Czech Republic in March 2015. These 
training sessions were sponsored by IEA and were mandatory to participate in the TIMSS assessment. 
Materials from these sessions along with additional materials constructed specifically for this purpose 
were used to train the scoring supervisors and scorers. Two additional scoring directors were hired for the 
grade 4 and grade 8 mathematics and science items. These scoring directors were trained by the scoring 
representative who attended the training in New Zealand. 


There were 11 scoring supervisors hired to train and oversee the scorers on the grade 4 and grade 8 
mathematics and science items. Eighty-three scorers organized into 11 teams, with 6 to 8 scorers per 
team, were hired to score these items. All were hired based on their experience with similar mathematics 
and science scoring projects. 


Two physics scoring supervisors and one advanced mathematics scoring supervisor were hired to train the 
scorers on the TIMSS Advanced items. Twenty-four scorers organized into three teams of 8 were hired to 
score the physics and advanced mathematics items. These supervisors and scorers were hired based on 
their background with physics and calculus. 


Training activities for the scorers followed the same routine as the international training, with supervisors 
leading each small team reading the item prompt; reading the rubric or scoring guide aloud; reading aloud 
each of the anchor papers and explaining the reasoning behind the score; allowing the scorers time to 
complete the practice papers; reviewing each of the practice papers; and opening individual scoring on the 
electronic Performance Evaluation Network (ePEN). Training on and scoring of the items was completed 
one at a time, with the exception of the linked items, which were scored together. 


Scoring 


Each team worked on different items. Scorers were able to view only the one active item that they were 
assigned by their supervisors and were not able to score other items from any other team. The exception 
was with the linked items from the “themed” blocks. In this situation, a question or questions required 
reference to a previous answer. For instance, question 4 in the block may have required data or the answer 
from questions 1 and 2 in order to answer the question. In this situation, the ePEN setup was designed so 
that the scorer had access to screens with all of the pertinent responses, again only as assigned by the 
supervisor. The answers could not be submitted to ePEN until all questions from the set were scored to 
make sure that the scorer had all the needed information to score all related items. 


Scoring quality was monitored continuously. The ePEN system allowed interrater reliability reports to be 
run almost as soon as scoring began. Another monitoring method used was back-reading of already 
scored responses. This allowed the scoring supervisor to look at responses by category. The scoring 
supervisor also could review responses either by scorer or by score point agreements or splits. Scoring 
supervisors checked completion statistics as well. 
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The United States scored approximately 366,900 grade 4 and grade 8 science student responses; 
approximately 364,868 grade 4 and 8 mathematics student responses; and approximately 52,700 physics 
and 51,300 advanced mathematics student responses for TIMSS Advanced. This included the number of 
responses selected for interrater reliability or second scoring. 


Cross-Country Scoring Reliability Study 


In international assessments, it is also important to gather information about how reliably the scoring was 
conducted from country to country so that valid international comparisons can be made of students’ 
achievement. To document the reliability of constructed-response scoring across TIMSS countries, a 
cross-country scoring reliability study was conducted. The Cross-country Scoring Reliability Study was 
introduced to estimate the degree of agreement between scorers in different countries. Cross-country 
reliability scoring allows scorers from each participating country to score the same set of student 
responses from other English speaking countries. 


The IEA Data Processing Center assembled a sample of student responses to be scored and distributed the 
sample to each participating education system along with the IEA Cross-Country Scoring Reliability 
Software (CCSRS). The sample responses involved 21 (11 mathematics and 10 science items) items at 
the fourth grade and 26 (13 mathematics and 13 science items) items at the eighth grade. All of the items 
were trend items from TIMSS 2011. After scoring the U.S. responses for an item with their team, two 
scorers then completed scoring of the international responses that were preloaded on desktop computers. 
While the individuals making up the pairs varied, no scorer worked on items that he or she had not been 
trained to score. Completion reports were only available with the CCSRS system. Once scoring was 
completed, the scored files were sent back to the IEA through a secure FTP site. 


Trend Reliability Scoring 


To document the reliability of constructed-response scoring between the TIMSS 2015 scorers and the 
TIMSS 2011 scorers in the United States, TIMSS included a trend scoring reliability study. The trend 
reliability scoring allows scorers of the TIMSS 2015 assessment to score a subset of student responses 
collected in the United States for TIMSS 2011. 


The IEA Data Processing Center assembled a sample of student responses to be scored and distributed it 
to each participating education system along with the IEA Trend Scoring Reliability Software (TSRS). 
Only education systems participating in 2015 that also participated in TIMSS 2011 took part in the trend 
scoring reliability study. There were 44 grade 4 items and 60 grade 8 items for TIMSS trend scoring. 


A total number of responses (200) were divided equally by the number of scorers who were assigned to 


the item block. The main trend reliability scoring process took place in parallel with the main study 
scoring for TIMSS 2015. 
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Data Entry 


The data in the TIMSS scannable student booklets was collected with an optical-scanning equipment that 
captured images of the open-ended constructed-response items, multiple-choice responses, and intelligent 
character recognition (ICR) fields. 


The data values captured from the multiple-choice items were coded as numeric data. Unmarked fields 
were coded as blanks and the editing staff was alerted to missing or encoded critical data. The images of 
open-ended constructed-response items were saved as a digitized computer file and uploaded into the 
ePEN scoring system for scoring. ICR was used to read various hand and machine printing in the 
documents. Image clips of the ICR fields were displayed to online editing staff for verification. In 
addition to capturing the student responses, the barcode identification numbers used to maintain process 
control were decoded and transcribed to the TIMSS computerized data file. 


Data Editing 


Each data set produced by the scanning system was validated for type and range of response. The data- 
entry and resolution system used was able to simultaneously process a variety of materials from all 
grades, subject areas, and assessment booklets as the materials were submitted to the system from 
scannable media. 


The data records in the scan file were organized in the same order in which the paper materials were 
processed by the scanner. As the program processed each record within a batch from the scan file, it wrote 
the edited and reformatted data records to the pre-edit file and recorded all errors on the edit file. The 
program generated an online edit file of the data problems and resolution guidelines. Image clips 
requiring edits were routed to online editing stations for the imaged scanned documents. 


All data values that were out of range were read “as is” but were flagged as suspect. All data fields that 
were read as asterisks (*) were recorded on the online edit file. Because the asterisk code indicated a 
double response, these items were identified for possible resolution by editing staff. Each field was 
validated for range response and any values outside of the specified range. Corrections were made 
immediately. The system employed an edit/verify system, which meant two different people viewed the 
same suspect data and operated on it separately. The verifier made sure the two responses (one from 
either the entry operator or the ICR engine) were the same before the system accepted the item as being 
correct. If it could not be determined, it was escalated to a supervisor. 


When the edit process produced an error-free file, the booklet ID number was posted to the TIMSS 
tracking file by grade and school. This permitted staff to monitor the TIMSS processing effort by 
accurately measuring the number of documents processed. The posting of booklet IDs also ensured that a 
booklet ID was not processed more than once. 


File Creation and Consistency Checks 


In a final step the data from the assessment score files were merged with the student scanned data. At this 
time, final output files were produced for each file type. The final files were checked to ensure the data 
were in the correct format. In earlier editing functions, data were checked for completeness and 


U.S. TIMSS 2015 and TIMSS Advanced 1995 & 2015 Technical Report and User’s Guide 4-31 


compliance with the international codebook specifications. In addition, a check was performed to verify 
correct linking and matching of student and school data files. The student data files were loaded in the 
WinW3sS software such that all data from the assessments and questionnaires were available in the format 
required by IEA. 


4.6 Data Preparation 


As noted in the previous section, the data collected for TIMSS and TIMSS Advanced 2015 were imported 
into data files according to a common international format, as specified in the IEA Data Management 
Expert—TIMSS & TIMSS Advanced 2015 MS data entry software. The software facilitated the checking 
and correction of data by providing various data consistency checks. 


The data files in this format were sent to the IEA Data Processing Center, where they were subjected to an 
extensive series of data cleaning and consistency checks. The overriding concern of these checks was to 
ensure that all information in the database conformed to the internationally defined data structure, 
national adaptations to questionnaires were reflected appropriately in the codebooks and documentation, 
and all variables used for international comparisons were comparable across countries. 


4.6.1 International Data Cleaning Procedures 


The IEA Data Processing Center was responsible for checking the data files from each country, applying 
standard cleaning rules to verify the accuracy and consistency of the data, and documenting electronically 
any deviations from the international file structure. Queries arising during this process were addressed to 
national centers, and this process was repeated as necessary to ensure the data were consistent and 
comparable within and between countries. 


Following this cleaning step, countries were provided national univariate and reliability statistics along 
with data almanacs containing international univariate statistics and item statistics. This allowed countries 
to examine their data with those of other participating nations. Once any problems arising from this 
examination were resolved, sampling weights produced by Statistics Canada and IRT-scaled student 
proficiency scores in mathematics and science were added to the file. 


Detailed information on the entire data entry and cleaning process can be found in Methods and 
Procedures in TIMSS 2015 (Martin, Mullis, and Hooper 2016a and in Methods and Procedures in TIMSS 
Advanced 2015 (Martin, Mullis, and Hooper 2016b. 


4.6.2 Data Confidentiality Safeguards 


While the National Center for Education Statistics (NCES) and data contractors routinely pledge 
confidentiality to respondents, concerns about the potential for disclosure of information about individual 
survey respondents have recently increased, particularly with software and online services available to 
individuals that could facilitate respondent identification. Data confidentiality laws have been enacted 
since the Privacy Act of 1974 to further ensure the protection of personally identifiable information (PI). 
The Education Sciences Reform Act of 2002 explicitly requires that NCES protect the confidentiality of 
all data collected from respondents in NCES-sponsored surveys. More specifically, NCES Standard 4-2, 
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Maintaining Confidentiality provides the guidelines for limiting the risk of data disclosure for data 
released by NCES (Seastrom 2002). Data disclosure occurs when an individual respondent has been 
identified through the use of the survey item responses and other external data sources. The procedures 
used to reduce the risk of data disclosure for TIMSS and TIMSS Advanced 2015, in accordance with the 
guidelines specified in NCES Standard 4-2, are described below. 


All students, teachers, and schools participating in TIMSS and TIMSS Advanced 2015 do so with the 
assurance that their identities will not be disclosed. All employees handling the data have signed 
affidavits of data confidentiality. The names of schools, students and teachers are removed by the field 
staff from the TIMSS questionnaires and assessment booklets, and are either physically cut or blacked out 
on the corresponding TIMSS forms. School, student, and teacher names are replaced by computer- 
generated school, student, and teacher IDs. Furthermore, any hard-copy questionnaires and assessment 
booklets are sealed by security stickers to ensure that neither school nor project field staff can access the 
questionnaire responses provided by the respondents. 


Additionally, the following NCES disclosure statement has been placed in all of the TIMSS 
questionnaires, including school and teacher questionnaires administrated online: 


U.S. participation in this study is sponsored by the National Center for Education 
Statistics (NCES), U.S. Department of Education. Your responses are protected 
by federal statute (P.L. 107-279, Title I, Part E, Sec.183). Your answers may be 
used only for statistical purposes and may not be disclosed, or used, in 
identifiable form for any other purpose except as required by law. By law, 
everyone working on this NCES survey is subject to a jail term of up to 5 years, 
a fine of up to $250,000, or both if he or she willfully discloses ANY identifiable 
information about you. 


After the TIMSS and TIMSS Advanced data was collected, procedures were implemented to ensure the 
confidentiality of the school, student, and teacher responses. The Institute of Education Sciences (IES) 
requires that survey data cannot be transmitted to foreign entities prior to ensuring the confidentiality of 
the data and approval by the IES Disclosure Review Board. The TIMSS and TIMSS Advanced disclosure 
analyses focused on the two Disclosure Review Board-required components for securing the 
confidentiality of the TIMSS data. They included (1) the identification and masking of potential 
disclosure-risk TIMSS schools by comparing the study variables with publicly available school files, 
using probabilistic matching and deterministic swapping on school-level data, and (2) the implementation 
of an additional measure of uncertainty of school, student, and teacher identification with the random 
swapping of data elements within the school, student, and teacher files. 


The confidentiality analyses were designed to provide reasonable assurance that the 2015 TIMSS U.S. 
public-use data files will not allow identification of individual schools, students, or teachers when 
compared to publicly available data collections. No publicly available data collections identify either 
students or teachers by name, but two data collections identify schools by name. The NCES regularly 
publishes the Common Core of Data (CCD), a detailed public school listing, and the Private School 
Survey (PSS), a detailed private school listing. Any potential identification of teachers and students will 
arise through the identification of their associated schools. Providing a reasonable degree of assurance 
that TIMSS schools cannot be identified thus ensures that teacher and student data also remain 
unidentifiable. Nevertheless, the proposed confidentiality procedures included perturbing student- and 
teacher-level data. It should be noted that the TIMSS data do not contain any information of a personal or 
sensitive nature (such as drug use or sexual activities). 
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Anyone intent on identifying the TIMSS schools could take the TIMSS public-use U.S. school data and 
search the CCD and PSS files for matching schools. For example, one could match a public school from 
the TIMSS U.S. school file against the CCD school file by first identifying variables common in both 
files. One then could compare a TIMSS school record to each CCD school record, assigning each pair a 
probability of being a true match based on the common variables. The CCD record (or records) with the 
highest probability would best match the traits of the TIMSS school. A TIMSS school could then be 
identified from the school name and address attached to the CCD school record. The user could take a 
similar approach in identifying a TIMSS private school from the PSS school file. 


Because the variables in the TIMSS and TIMSS Advanced 2015 data files were obtained from responses 
to the school questionnaire, which often vary from the CCD and PSS data, exact profile matches are 
unlikely. Even then, one would not know for certain whether any of the matched schools were the actual 
TIMSS and TIMSS Advanced schools or whether the match had simply arisen by chance. Nevertheless, 
school matching analyses were undertaken using probabilistic matching algorithms approved by the IES 
Disclosure Review Board for use in disclosure analyses. These algorithms identified schools whose 
school questionnaire responses provided some potential for identification. To provide further protection, 
elements of the data from schools identified as “disclosure risks” in this way were perturbed using the 
procedures approved by the IES Disclosure Review Board. After perturbation, the data were subjected to 
another round of analyses to ensure that the potential for identification no longer existed. 


An additional measure was taken to reduce further the risk of disclosure of an individual respondent. This 
random data swapping measure is an IES Disclosure Review Board requirement that reduces disclosure 
risk by modifying microdata. In random data swapping, a probability sample of records is paired with 
other records on the file using selected characteristics, and then some identifying variables are swapped 
between the pairs of records (Kaufman, Seastrom, and Roey 2005). The sampling rate and variables for 
TIMSS and TIMSS Advanced swapping were designed to protect the confidentiality of the data without 
affecting the usability of the dataset. All questionnaire data (school, teacher, and student) were involved 
in the swapping. This method is an effective way of keeping as much valuable data as possible while 
protecting respondent identity. Swapping preserves the univariate frequencies, means, and variances, 
although it may slightly affect multivariate relationships. Pre- and post-swapping percentage distributions 
(unweighted and weighted) and correlations were reviewed to ensure data quality was maintained. 


Confidentiality analyses of this kind were conducted before the U.S. data files were delivered to the Data 
Processing Center for cleaning, and prior to the score scaling and estimation of sampling weights. 


Because of increasing security concerns from the international community, an additional data 
confidentiality measure was implemented by the IEA DPC for both TIMSS and TIMSS Advanced 2015. 
School IDs were scrambled for each education system’s participating schools as a security measurement 
for de-identification. This procedure puts an additional safeguard into place that makes it difficult to trace 
collected information back to the source. Because of the hierarchical ID naming convention employed by 
IEA DPC, the scrambling of the school IDs also affected the teacher and student IDs, thus making these 
IDs scrambled as well. 


4.6.3 The Estimation of TIMSS and TIMSS Advanced Student Proficiencies 


All cycles of TIMSS and TIMSS Advanced used item response theory (IRT) methods to produce score 
scales that summarized the achievement results. With this method, the performance of a sample of 
students in a subject area or sub-area could be summarized on a single scale or a series of scales, even 
when different students had been administered different items. 
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IRT scaling provides estimates of item parameters (for example, item difficulty and item discrimination) 
that define the relationship between the item and the underlying variable measured by the test. Parameters 
of the IRT model are estimated for each test item, with an overall scale being established as well as scales 
for each content area and cognitive domain specified in the assessment framework. 


To allow for the calculation of trends in achievement, comparisons of scores across the six TIMSS 
assessments conducted in 1995, 1999, 2003, 2007, 2011 and 2015 and comparisons of scores across the 
TIMSS Advanced assessments conducted in 1995 and 2015 were necessary. To this end, achievement 
scores from all five TIMSS assessments were placed on the same scale. The three TIMSS Advanced 
cycles were subjected to the same treatment. Details are provided in Methods and Procedures in TIMSS 
2015 (Martin, Mullis, and Hooper 2016a) and in Methods and Procedures in TIMSS Advanced 2015 
(Martin, Mullis, and Hooper 2016b). 


Plausible Values 


During the scaling phase, plausible values were used to characterize scale scores for students participating 
in the assessment. 


To keep student burden to a minimum, TIMSS and TIMSS Advanced purposefully administered a limited 
number of assessment items to each student—too few to produce accurate individual content-related scale 
scores for each student. The number of assessment items administered to each student, however, is 
sufficient to produce accurate group content-related scale scores for subgroups of the population. These 
scores are transformed during the scaling process into plausible values to characterize students 
participating in the assessment, given their background characteristics. Plausible values are imputed 
values and not test scores for individuals in the usual sense. If used individually, they provide biased 
estimates of the proficiencies of individual students. However, when grouped as intended, plausible 
values provide unbiased estimates of population characteristics (e.g., means and variances for groups). 


Plausible values represent what the performance of an individual on the entire assessment might have 
been, had it been observed. They are estimated as random draws (usually five) from an empirically 
derived distribution of score values based on the student's observed responses to assessment items and on 
background variables. Each random draw from the distribution is considered a representative value from 
the distribution of potential scale scores for all students in the sample who have similar background 
characteristics and similar patterns of item responses. Differences between plausible values drawn for a 
single individual quantify the degree of error (the width of the spread) in the underlying distribution of 
possible scale scores that could have caused the observed performances. 


This approach to the estimation of scale scores ensures that the estimates of the average performance of 
student populations and the estimates of variability in those estimates are more accurate than those 
determined through traditional procedures, which estimate a single score for each student. An accessible 
treatment of the derivation and use of plausible values can be found in Beaton and Gonzalez (1995). A 
more technical treatment can be found in Methods and Procedures in TIMSS 2015 (Martin, Mullis, and 
Hooper 2016a) and in Methods and Procedures in TIMSS Advanced 2015 (Martin, Mullis, and Hooper 
2016b). 
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International Benchmarks 


International achievement benchmarks were developed to provide a concrete interpretation of what the 
scores on the TIMSS mathematics and science achievement scales and on the TIMSS Advanced physics 
and advanced mathematics scales mean. TIMSS and TIMSS Advanced each used scale anchoring to 
summarize and describe student achievement at points on the mathematics and science (for TIMSS), and 
advanced mathematics and physics (for TIMSS Advanced) scales. TIMSS identifies four benchmarks: 
Advanced (625), High (550), Intermediate (475), and Low (400). TIMSS Advanced has the same top three 
benchmarks but no Low benchmark. Scale anchoring involves selecting benchmarks (scale points) on the 
TIMSS and TIMSS Advanced achievement scales to be described in terms of student performance and 
then identifying items that students scoring at the anchor points can answer correctly. Subsequently, these 
items are grouped by content area within benchmarks and reviewed by mathematics and science experts 
for TIMSS grades 4 and 8, and advanced mathematics and physics experts for TIMSS Advanced. These 
experts focus on the content of each item and describe the kind of mathematics, science, advanced 
mathematics, or physics knowledge demonstrated by students answering the item correctly. The experts 
then provide a summary description of performance at each anchor point leading to a content-referenced 
interpretation of the achievement results. Detailed information on the creation of the benchmarks is 
provided in the “Using Scale Anchoring to Interpret the TIMSS and TIMSS Advanced 2015 Achievement 
Scales” section of Methods and Procedures in TIMSS 2015 (Martin, Mullis, and Hooper 2016a) and in 
Methods and Procedures in TIMSS Advanced 2015 (Martin, Mullis, and Hooper 2016b). 


4.6.4 The Estimation of Sampling Weights 


Because of the complex sampling design used in TIMSS and TIMSS Advanced, students were assigned 
sampling weights. In general the sampling weight assigned to a student is the inverse of the probability 
that the student would be selected for the sample. When responses are weighted, each contributes to the 
results for the total number of students represented by the individual student assessed. Weighting also 
adjusts for school and student nonresponse. The internationally defined weighting specifications for 
TIMSS and TIMSS Advanced require that each assessed student’s sampling weight should be the product 
of the following: 


rT the inverse of the school’s probability of selection; 

a an adjustment for school-level nonresponse; 

7 the inverse of the classroom’s probability of selection; and 
a an adjustment for student-level nonresponse. 


Sampling weights should be used in all TIMSS and TIMSS Advanced analyses. A detailed description of 
this process is provided in TIMSS 2015 User Guide for the International Database (Foy 2017a) and the 
TIMSS Advanced 2015 User Guide for the International Database (Foy 2017b). 
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5. TIMSS AND TIMSS ADVANCED 2015 DATA 
FOR THE UNITED STATES AND FLORIDA 


The TIMSS and TIMSS Advanced 2015 international databases, released by the TIMSS and PIRLS 
International Study Center, contain student achievement data as well as student, teacher, school, and 
curricular background data for 47 countries and 6 other education systems at grade 4 and for 39 countries 
and 6 other education systems at grade 8 for TIMSS, including Florida as a benchmarking state for the 
U.S.; and for 9 countries for TIMSS Advanced 2015. The following five basic types of data files are 
available for each education system in both the TIMSS and TIMSS Advanced international datasets: 


1. achievement files containing item response data and scale scores for the TIMSS or 
TIMSS Advanced assessment; 


2. background files with information from students, from their mathematics and science 
teachers in TIMSS or their advanced mathematics and physics teachers in TIMSS 
Advanced, and from the principals of their schools; 


3. student-teacher linkage files that contain the information needed to link data on 
students to that of their teachers; 


4. constructed-response scoring reliability files providing data on the reliability of 
scoring for this type of item; and 


5. curriculum data files that contain the responses of countries or participating education 
systems to the curriculum questionnaires. 


These databases provide comparable data across education systems on detailed measures of student 
achievement in mathematics and science for TIMSS participating education systems, '’ detailed measures 
of achievement in advanced mathematics and physics for TIMSS Advanced participants; information on 
educational practices and student outcomes; linking variables between student achievement and 
background information from students, teachers, school principals, and curriculum experts; and 
achievement scales on a metric that is common to all cycles of TIMSS and TIMSS Advanced 
respectively, allowing for the analysis of trends. 


The international data files are available in two versions: a public-use version and a restricted-use version. 
The public-use data files are available as SAS export files or SPSS “.sav” files through 
https://timssandpirls.bc.edu/timss2015/international-database/ (TIMSS) or 
https://timssandpirls.bc.edu/timss2015/advanced-international-database/ (TIMSS Advanced). The 
restricted-use data files are available by request through the IEA Study Data Repository 
(http://www.iea.nl/data.html). Discussions of these types of data files can be found in the TIMSS 2015 
User Guide for the International Database (Foy 2017a) or the TIMSS Advanced 2015 User Guide for the 
International Database (Foy 2017b). These are the most comprehensive and detailed references for the 
TIMSS and TIMSS Advanced 2015 data and should be seen as the primary references. 


'l Data for Florida are not included in the international datasets due to data confidentiality issues. 
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The U.S. international data files, which are part of the TIMSS and PIRLS International Study Center 
international databases, do not include the U.S.-specific adaptations made to a few questions in the 
questionnaires or the additional questions added to the school and student questionnaires, such as the 
question on race/ethnicity added to the student questionnaire. In addition, data for Florida are not included 
in the international data files due to potential confidentiality issues. 


For users who are only interested in TIMSS and TIMSS Advanced 2015 data pertaining to U.S. and/or 
Florida, NCES releases the U.S.-specific and Florida data files separately from the international 
databases mentioned above. In addition to the TIMSS Advanced 2015 data files, a set of the TIMSS 
Advanced 1995 school and student data files are also provided by NCES for trend analysis purposes. This 
chapter focuses on the contents of the NCES-released data files. 


5.1 U.S.-Specific and Florida Data Files Released by NCES 


Similar to the TIMSS and PIRLS International Study Center, NCES released the data in two versions: 


7 public-use data files, which include 1) TIMSS and TIMSS Advanced 2015 U.S. 
national data, and 2) TIMSS Advanced 1995 U.S. national data. These data are 
available by download from https://nces.ed.gov/timss/datafiles.asp. The data files are 
in ASCII format and are named as indicated in Exhibit 5-1. SAS and SPSS codes for 
reading these data files can also be downloaded from the NCES website. '* 


a restricted-use data files, which include 1) TIMSS and TIMSS Advanced 2015 U.S. 
national data, 2) TIMSS Advanced 1995 U.S. national data, and 3) Florida 2015 data. 
These data can only be obtained by completing a restricted-use license agreement with 
NCES. The restricted-use data files are provided only on CD-ROM or DVD-ROM. 
These datasets contain the supplemental link files that link TIMSS or TIMSS 
Advanced school ID numbers to the school ID numbers as they appear in the publicly 
available Common Core of Data (CCD) or the Private School Universe Survey (PSS). 
In addition, race/ethnicity is provided with all available categories and free or 
reduced-price lunch is provided as a continuous variable. Because these data can 
reveal the identities of participating schools, the restricted-use data files are only made 
available to those who obtain a NCES restricted-use data license. Directions on how 
to obtain the license can be found at http://nces.ed.gov/pubsearch/licenses.asp. 

Exhibit 5-2 lists the restricted-use data files released by NCES. 


"The public-use dataset and restricted-use data CD contain a quick guide that explains how to use ASCII files with control code files to create 
SPSS or SAS datasets. 
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Exhibit 5-1. 
public-use version 


Contents 


TIMSS 2015: Grade 4 
Student Background 
Student Achievement 
Teacher Background 
Student-Teacher Link 
School Background 


TIMSS 2015: Grade 8 
Student Background 
Student Achievement 
Math Teacher Background 
Science Teacher Background 
Student-Teacher Link 
School Background 


TIMSS Advanced 2015: Advanced Mathematics 
Student Background 
Student Achievement 
Advanced Mathematics Teacher Background 
Student-Teacher Link 
School Background 


TIMSS Advanced 2015: Physics 
Student Background 
Student Achievement 
Physics Teacher Background 
Student-Teacher Link 
School Background 


TIMSS Advanced 1995: Advanced Mathematics” 
Student Background 
Student Achievement 
School Background 


TIMSS Advanced 1995: Physics” 
Student Background 
Student Achievement 
School Background 


U.S. TIMSS 2015 and TIMSS Advanced 1995 & 2015 data files released by NCES, 


File name 


T4_STUDENT15.DAT 
T4_ACHIEVE15.DAT 
T4_TEACHER15.DAT 
T4_STD_TCH_LINK15.DAT 
T4_SCHOOL15.DAT 


T8_STUDENT15.DAT 
T8_ACHIEVE15.DAT 
T8_MTEACHERI15.DAT 
T8_STEACHER15.DAT 
T8_STD_TCH_LINK15.DAT 
T8_SCHOOL15.DAT 


TA_M_STUDENT15.DAT 
TA_M_ACHIEVE15.DAT 
TA_M_MTEACHER15.DAT 
TA_M_STD_TCH_LINK15.DAT 
TA_M_SCHOOL15.DAT 


TA_P_STUDENT15.DAT 
TA_P_ACHIEVE15.DAT 
TA_P_PTEACHER15.DAT 
TA_P_STD_TCH_LINK15.DAT 
TA_P_SCHOOL15.DAT 


TA_M_STUDENT95.DAT 
TA_M_ACHIEVE95.DAT 
TA_M_SCHOOL95.DAT 


TA_P_STUDENT95.DAT 
TA_P_ACHIEVE95.DAT 
TA_P_SCHOOL95.DAT 


“There was no teacher questionnaire administered to the teachers of students in their final year of secondary school for TIMSS Advanced 
1995; therefore no teacher data files are provided for TIMSS Advanced 1995. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Trends in International Mathematics and Science Study 
(TIMSS), 2015. 
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Exhibit 5-2. U.S. TIMSS 2015, TIMSS Advanced 1995 & 2015, and Florida data files released by 


NCES, restricted-use version 


Contents 


TIMSS 2015: Grade 4 Florida 
Student Background 
Student Achievement 
Teacher Background 
Student-Teacher Link 
School Background 


TIMSS 2015: Grade 8 Florida 
Student Background 
Student Achievement 
Mathematics Teacher Background 
Science Teacher Background 
Student-Teacher Link 
School Background 


TIMSS 2015: Grade 4 Restricted-Use 
Restricted-use School Variables 
Restricted-use Student Variables 


TIMSS 2015: Grade 8 Restricted-Use 
Restricted-use School Variables 
Restricted-use Student Variables 


TIMSS Advanced 2015: Advanced Mathematics 
Restricted-use School Variables 
Restricted-use Student Variables 


TIMSS Advanced 2015: Physics 
Restricted-use School Variables 
Restricted-use Student Variables 


TIMSS Advanced 1995: Advanced Mathematics 
Restricted-use School Variables 
Restricted-use Student Variables 


TIMSS Advanced 1995: Physics 
Restricted-use School Variables 
Restricted-use Student Variables 


File name 


FL_T4_STUDENT15.DAT 
FL_T4_ACHIEVE15.DAT 
FL_T4_TEACHER15.DAT 
FL_T4_STD_TCH_LINK15.DAT 
FL_T4_SCHOOL15.DAT 


FL_T8_STUDENT15.DAT 
FL_T8_ACHIEVE15.DAT 
FL_T8_MTEACHER15.DAT 
FL_T8_STEACHER15.DAT 
FL_T8_STD_TCH_LINK15.DAT 
FL_T8_SCHOOL15.DAT 


T4_RESTRICTED_USE15_SCH.DAT 
T4_RESTRICTED_USE15_STUD.DAT 


T8_RESTRICTED_USE15_SCH.DAT 
T8_RESTRICTED_USE15_STUD.DAT 


TA_M_RESTRICTED_USE15_SCH.DAT 
TA_M_RESTRICTED_USE15_STUD.DAT 


TA_P_RESTRICTED_USE15_SCH.DAT 
TA_P_RESTRICTED_USE15_STUD.DAT 


TA_M_RESTRICTED_USE95_SCH.DAT 
TA_M_RESTRICTED_USE95_STUD.DAT 


TA_P_RESTRICTED_USE95_SCH.DAT 
TA_P_RESTRICTED_USE95_STUD.DAT 


SOURCE: U.S. Department of Education, National Center for Education Statistics, Trends in International Mathematics and Science Study 


(TIMSS), 2015. 
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5.2 TIMSS and TIMSS Advanced 2015 Student Achievement Data 
Files and Variable Names 


The data files contain the IRT-scaled achievement scores for overall mathematics and science/physics, as 
well as scores for the several mathematics and science/physics content domains and the three mathematics 
and science/physics cognitive domains. A set of five plausible values characterizes each of these 
achievement scores. 


For analytic convenience, these same achievement scores are also provided as an addition to the student 
background data files. 


Exhibits 5-3, 5-4, and 5-5 show the nomenclature for identifying the various overall content domain and 
cognitive domain achievement scale score variables, each of which is represented by five plausible 
values. The achievement score variable names are based on an eight-character string defined below. In 
exhibit 5-3, these conventions are illustrated by reference to the first plausible value for each of the total, 
content domain, and cognitive domain achievement scales at each grade level for TIMSS. In exhibits 5-4 
and 5-5, these conventions are illustrated by reference to the first plausible value for each of the total, 
content domain, and cognitive domain achievement scales at each grade level for TIMSS Advanced. 
First character of the variable name (ASMMATO01): 

7 A-—TIMSS 2015 fourth-grade score 

0 B-TIMSS 2015 eighth-grade score 

7 M —TIMSS Advanced 2015: advanced mathematics score 

P—TIMSS Advanced 2015: physics score 


Second character of the variable name (ASMMATO01): 


5 S — indicates that this is a student score variable 
Third character of the variable name (ASMMAT01): 
rT M -—TIMSS 2015 mathematics score 
7 S— TIMSS 2015 science score 
Hi M-—TIMSS Advanced 2015: advanced mathematics score 
a P—TIMSS Advanced 2015: physics score 


Fourth through sixth characters of the variable name (ASMMAT01): 


. Three-character code identifying the achievement scale, as indicated in exhibits 5-3, 
5-4, and 5-5 below. 
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Seventh and eighth characters of the variable name (ASMMAT01): 


01 — first plausible value 
02 — second plausible value 
03 — third plausible value 
04 — fourth plausible value 


05 — fifth plausible value 


For example, ASMMAT01 is the first plausible value for the fourth-grade mathematics total score, while 
ASSSCIO02 is the second plausible value for the total science score. ASMNUM603 is the third plausible 
value for the mathematics content domain “Number,” and ASSEAR04 is the fourth plausible value for the 
science content domain “Earth science.” Similarly, ASMKNOO01 is the first plausible value for the 
mathematics cognitive domain “Knowing,” and ASSAPP05 is the fifth plausible value for the science 


cognitive dom 


ain “Applying.” 


Exhibit 5-3. Three-digit codes identifying achievement scales in TIMSS 2015 
Score Examples of variable names 
identifier Grade 4 Grade 8 

Mathematics overall score MAT ASMMATO1 BSMMATO1 
Science overall score SCI ASSSCIO01 BSSSCIO01 
Mathematics content domains 

Number NUM ASMNUMO1 BSMNUMOL1 

Geometric Shapes/Geometry GEO ASMGEOO01 BSMGEO01 

Data Display/Data and Chance DAT ASMDAT01 BSMDATO1 

Algebra ALG BSMALGO1 
Science content domains 

Life Science LIF ASSLIFO1 

Physical Science/Physics PHY ASSPHY01 BSSPHY01 

Earth Science EAR ASSEARO1 BSSEARO1 

Chemistry CHE BSSCHEO1 

Biology BIO BSSBIOO1 
Mathematics cognitive domains 

Knowing KNO ASMKNOOL1 BSMKNOOL1 

Applying APP ASMAPPO1 BSMAPPO1 

Reasoning REA ASMREAO1 BSMREAO1 
Science cognitive domains 

Knowing KNO ASSKNOO1 BSSKNOOL1 

Applying APP ASSAPPO1 BSSAPPO1 

Reasoning REA ASSREAO1 BSSREAOL1 


a tt 
SOURCE: Foy, P. (2017a). TIMSS 2015 User Guide for the International Database. Chestnut Hill, MA: TIMSS & PIRLS International Study 


Center, Boston Coll 
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Exhibit 5-4. Three-digit codes identifying achievement scales in TIMSS Advanced: Advanced 


mathematics 
Examples of variable names 
Score identifier TIMSS Advanced Mathematics 
Advanced mathematics overall score MAT MSMMATO1 
Mathematics content domains 

Algebra ALG MSMALGO01 
Calculus CAL MSMCALO1 
Geometry GEO MSMGEO01 

Mathematics cognitive domains 
Knowing KNO MSMKNOO1 
Applying APP MSMAPPO01 
Reasoning REA MSMREAO1 


SOURCE: Foy, P. (2017b). TIMSS Advanced 2015 User Guide for the International Database. Chestnut Hill, MA: TIMSS & PIRLS 
International Study Center, Boston College. 


Exhibit 5-5. Three-digit codes identifying achievement scales in TIMSS Advanced: Physics 


Examples of variable names 
Score identifier TIMSS Advanced Physics 


Physics overall score PHY PSPPHY01 
Physics content domains 
Electricity and Magnetism ELE PSPELEO1 
Mechanics and Thermodynamics MEC PSPMECO01 
Wave Phenomena and Atomic Nuclear Physics WAV PSPWAVO1 
Physics cognitive domains 
Knowing KNO PSPKNOO1 
Applying APP PSPAPPO1 
Reasoning REA PSPREAO1 


SOURCE: Foy, P. (2017b). TIMSS Advanced 2015 User Guide for the International Database. Chestnut Hill, MA: TIMSS & PIRLS 
International Study Center, Boston College. 


5.2.1 TIMSS and TIMSS Advanced 2015 Benchmark Achievement Variables 


The TIMSS and TIMSS Advanced achievement files also contain a set of variables indicating which 
international benchmark the students reached. For TIMSS, the overall mathematics and science scales at 
both grades have five plausible values defined as the international benchmark level reached (Advanced, 
High, Intermediate, and Low). Similarly, for TIMSS Advanced, the overall advanced mathematics and 
physics scales have five plausible values defined as the international benchmark level reached (Advanced, 
High, and Intermediate). The international benchmark variables follow the achievement score variable 
naming convention but substitute the letters “IBM” in the fourth through sixth positions of the variable 
name. Therefore, ASMIBM01, ASMIBM02, ASMIBM03, ASMIBMO04, and ASMIBM05 are the five 
benchmark variables describing the fourth-grade overall mathematics score. Similarly, BSMIBMO1 
through BSMIBM05 are the five benchmark variables describing the eighth-grade overall mathematics 
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score, and BSSIBMO01 through BSSIBMO05 are the five benchmark variables describing the eighth-grade 
overall science score. For TIMSS Advanced, MSMIBM01 through MSMIBM05 are the five benchmark 
variables describing the overall advanced mathematics score, and PSPIBMO01 through PSPIBMO05 are the 
five benchmark variables describing the overall physics scores. Details for TIMSS are provided in the 
TIMSS 2015 User Guide for the International Database (Foy 2017a, p. 57) and for TIMSS Advanced in 
TIMSS Advanced 2015 User Guide for the International Database (Foy 2017b, p. 51-52). 


5.3 TIMSS and TIMSS Advanced 2015 Background Questionnaire 
Data Files 


Student, teacher, and school files contain the responses to the questions contained in the respective 
background questionnaires administered in TIMSS and TIMSS Advanced 2015, along with a file used to 
link the student and teacher background data appropriately when student and teacher files are merged. 


5.3.1 TIMSS and TIMSS Advanced 2015 Student Background Data Files 


The student background data files contain students’ responses to questions in the student questionnaire 
along with students’ mathematics and science achievement scores (as plausible values). At the fourth 
grade, there was a single version of the student questionnaire for TIMSS. There were two versions of the 
student questionnaire at the eighth grade for TIMSS: a version for education systems in which science is 
taught as an integrated subject (general science version), and another version for education systems in 
which the sciences (biology or life science, physics, chemistry, and Earth science) are taught separately. 
For eighth-grade students who were administered the general science version, as was the case for the 
United States and Florida, questions that appeared only in the separate science version were coded as “not 
administered.” For students in those education systems assigned the separate science versions, questions 
asked only in the general science version were coded as “not administered.” For TIMSS Advanced, there 
were two versions of the student questionnaire administrated: one for advanced mathematics and one for 
physics. 


The student background data files also contain a number of identification variables, tracking variables, 
sampling and weighting variables, and derived variables that were used to produce some of the exhibits in 
the international reports. 


5.3.2 TIMSS and TIMSS Advanced 2015 Teacher Background Data Files 


The mathematics and science teachers of the students sampled for TIMSS 2015 were administered at least 
one questionnaire for each TIMSS class taught. The teacher background data files contain one record for 
each of the classes taught. If teachers taught more than one class, they were expected to complete only 
one set of general background questions (part A), irrespective of the number of classes taught, and a 
separate part B (class-specific questions) for each class they taught. 


Responses to the single questionnaire administered to fourth-grade teachers of TIMSS are found in the 
teacher background data files. Separate TIMSS teacher questionnaires were administered to eighth-grade 
mathematics and science teachers. The responses of teachers to the mathematics questionnaire are found 
in the mathematics teacher background data files, and the responses of teachers to the science 
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questionnaire are in the science teacher background data files. Variable names for questions repeated in 
both questionnaires are the same. 


For TIMSS Advanced, teachers who taught advanced mathematics classes were administered the 
advanced mathematics teacher background questionnaire that had questions about their background and 
their teaching practices in the classes of the sampled students. Similarly, teachers who taught physics 
classes were administered the physics teacher background questionnaire that had questions about their 
background and their teaching practices in the classes of the sampled students. The responses of teachers 
to the advanced mathematics questionnaire are found in the advanced mathematics teacher background 
data files, and the responses of teachers to the physics questionnaire are in the physics teacher background 
data files. 


In all of the teacher files, each teacher has a unique identification number (IDTEACH) and a link number 
(IDLINK) specific to the class taught by the teacher and to which the information in the data record 
corresponds. The IDTEACH and IDLINK combination uniquely identifies, within an education system, a 
teacher teaching a specific class. For example, students linked to teachers identified by the same 
IDTEACH but different IDLINK are taught by the same teacher but in different classes. It is important to 
note that the teachers in question do not constitute a representative sample of teachers in an education 
system but rather are the teachers who taught a representative sample of students. To reflect this fact, for 
the most part, the teacher data should be analyzed only in conjunction with the student-teacher linkage 
data files and weighted with student sampling weights. 


5.3.3 TIMSS and TIMSS Advanced 2015 School Background Data Files 


The school background data files contain the responses of school principals to questions about school 
policy, resources, and environment asked in the school questionnaire. That file also contains a series of 
identification variables, link variables, and sampling variables. The school data files can be merged with 
the student data files by using the education system and school identification variables. Details of the 
merging procedure using the SPSS-linked IEA International Database (IDB) Analyzer or using SAS 
programs for TIMSS are described in the TIMSS 2015 User Guide for the International Database (Foy 
2017a); for TIMSS Advanced, these details are provided in the TIMSS Advanced 2015 User Guide for the 
International Database (Foy 2017b). 


5.3.4 TIMSS and TIMSS Advanced 2015 Student-Teacher Linkage Data Files 


The TIMSS and TIMSS Advanced 2015 student-teacher linkage data files contain information required to 
link the student and teacher data files. These files contain one entry per student-teacher linkage 
combination in the data. For instance, if three teachers are linked to a student, there are three entries in the 
file corresponding to that student. The sole purpose of the student-teacher linkage data files is to link 
teacher-level data with student-level data to perform appropriate student-level analyses where teacher 
characteristics are disaggregated over students. 
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5.3.5 TIMSS and TIMSS Advanced 2015 Curriculum Questionnaire Data Files 


In addition to the background questionnaires, TIMSS and TIMSS Advanced 2015 provide data on the 
curriculum of the participating education systems. For TIMSS there is a separate file for each grade, while 
for TIMSS Advanced there is a separate file for advanced mathematics and one for physics. 


5.4 Variable Naming Conventions for Background Variables 


The background variable naming convention is based on a seven- or eight-character string defined below. 
These conventions are illustrated by reference to an item in the fourth-grade school questionnaire. This 
item asks principals to report the population size of the community in which the school is located. 


First character of the variable name (ACBGOSA): 
0 A-—TIMSS 2015 fourth grade 
r B-TIMSS 2015 eighth grade 
7 M —TIMSS Advanced 2015: advanced mathematics 


7 P—TIMSS Advanced 2015: physics 


Second character of the variable name (ACBG05A): 


: C - school principal 
. T — teacher 
= S — student 


Third character of the variable name (ACBGOSA): 


Ey N —a nationally adapted background variable 

a X — anew national background variable 

7 B — all international background variables in the questionnaire data files 
r D — all international derived variables 


Fourth character of the variable name’? (ACBGO5A): 


mw G — general question (not subject specific) 


'3The letters “B,” “C,” and “E,” and “P” are used in the eighth-grade student background data files for variables corresponding to questions about 
separate sciences asked in the separate science version of the student questionnaire. “P” is also used in TIMSS Advanced: Physics student 
background data files for group identification. The letter “M” is used in the fourth- and eighth-grade student background data files for variables 
corresponding to questions about mathematics. It is also used in the TIMSS Advanced: Advanced Mathematics student background data files for 
group identification. 
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7 M — question related to mathematics or advanced mathematics 


a S — question related to science 

7 B — question related to biology or life science 

4 C — question related to chemistry 

: E — question related to Earth science 

: P — question related to physical science or physics 


Fifth through eighth characters of the variable name (ACBGOSA): 


: Used to represent the sequential numbering of each question 


5.4.1 Summary Indices and Derived Variables 


The TIMSS and TIMSS Advanced 2015 questionnaires often devote several questions to a single 
construct. In these cases, responses to the individual items were combined to create a derived variable. A 
TIMSS or TIMSS Advanced index is a special type of derived variable that assigns students to one of 
three levels— high, medium, or low—on the basis of their responses to the component variables. These 
variables are described in detail in Supplement 3 of the TIMSS 2015 User Guide for the International 
Database (Foy 2017a) and in Supplement 3 of the TIMSS Advanced 2015 User Guide for the 
International Database (Foy 2017b). 


5.4.2 Specific Variable Naming Conventions for TIMSS Advanced 1995 
Background Variables 


Similar to TIMSS and TIMSS Advanced 2015 variables, the naming convention for the TIMSS Advanced 
1995 background variables permits the identification of the population and questionnaire from which it 
was obtained based on 7- or 8-digit codes. However, there are some slight differences. The general 
definitions of the variable naming conventions for TIMSS Advanced 1995 are indicated below: 


First character of the variable name (MC1GCOMM/PC1GCOMM): 


a M-—TIMSS Advanced 1995: Mathematics 


P—TIMSS Advanced 1995: Physics 


Second character of the variable name (MC1GCOMM/PC1GCOMM): 


. C - school principal 


| S — student 
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Third character of the variable name (MC1GCOMM/PC1GCOMM): 


: N — a nationally adapted background variable 

iy X — anew national background variable 

a 1 — all international background variables in the questionnaire data files 
rT D — all international derived variables 


Fourth character of the variable name (MC1GCOMM/PC1GCOMM): 


. G — general question 

a M — question specific to mathematics 

a S — question specific to general science 
5 B — question specific to biology 

a C — question specific to chemistry 

7 E — question specific to Earth science 
a P — question specific to physics 


Fifth through eighth characters of the variable name (MC1GCOMM/PC1GCOMM): 


r Used to represent the abbreviated question reference 


5.5 Sampling and Weighting Variables 


Several sampling and weighting variables are included in the TIMSS and TIMSS Advanced 2015 data 
files. They are listed and described below in conjunction with a discussion of how and when these 
weights are used. Because TIMSS and TIMSS Advanced 2015 use a complex sampling design, sampling 
weights need to be used to generate accurate population estimates. The sampling weights account for the 
sample design, any stratification or disproportional sampling of subgroups, and also include adjustments 
for nonresponse (see Joncas and Foy 2012). 


As noted, the sample of students is not a simple random sample and, consequently, students in the sample 
do not have an equal probability of selection. Sampling weights adjust for this unequal probability and, in 
so doing, provide for statistical estimates reflective of the student population from which the sample was 
drawn. Sampling weights also include adjustments for school and student nonresponse. All TIMSS and 
TIMSS Advanced analyses require the application of sampling weights. Provisions for weighting data are 
a standard feature of virtually all software likely to be used in analyses. 


The sampling weights included in the TIMSS and TIMSS Advanced 2015 data files are described in 
exhibits 5-6 and 5-7. Note that teacher background data files do not have any sampling weight variables 
and teacher weights are located in the student-teacher linkage files. Analyses with teacher data are 
weighted properly by merging the teacher files with the student-teacher linkage files as described later in 
this chapter under section 5.11.5, Special Considerations in Using the Teacher Data. 
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Exhibit 5-6. TIMSS 2015 sampling weight variables 


Variable names | Descriptions 

TOTWGT Total student weight — sums to the national population 

SENWGT | Student senate weight — sums to 500 in each education system 

HOUWGT | Student house weight — sums to the student sample size in each education system 
TCHWGT | Overall teacher weight 

MATWGT | Mathematics teacher weight 

SCIWGT | Science teacher weight 

SCHWGT 'School-level weight 


SOURCE: Foy, P. (2017a). TIMSS 2015 User Guide for the International Database. Chestnut Hill, MA: TIMSS & PIRLS International Study 
Center, Boston College. 


Exhibit 5-7. TIMSS Advanced 2015 sampling weight variables 


Variable names ; Descriptions 

TOTWGT Total student weight — sums to the national population 

SENWGT | Student senate weight — sums to 500 in each education system 

HOUWGT | Student house weight — sums to the student sample size in each education system 
MATWGT Advanced mathematics teacher weight 

PHYWGT | Physics teacher weight 

SCHWGT School-level weight 


SOURCE: Foy, P. (2017b). TIMSS Advanced 2015 User Guide for the International Database. Chestnut Hill, MA: TIMSS & PIRLS 
International Study Center, Boston College. 


The characteristics of TIMSS and TIMSS Advanced 2015 sampling weight variables are as follows: 


m1 TOTWGT sums to the student population size in each education system and is 
appropriate for “within-country” 4 analyses and “cross-country” analyses where the 
analyses are conducted “country-by-country” and compared. 


r SENWGT is a transformation of TOTWGT that results in a weighted student sample 
size of 500 in each education system. This weight may be appropriate for “cross- 
country” analyses that require each education system to have the same number of 
students, rather than proportionately more students from larger education systems and 
fewer from smaller education systems, which is the case if TOTWGT is used. 


HOUWGT, another transformation of TOTWGT, ensures that the weighted sample 
corresponds to the actual sample size in each education system. This weight can be 
important since TOTWGT inflates sample sizes to approximate the population size, 
and software systems that use the actual sample size to compute significance tests 
would give misleading results under these conditions. 


4a |though traditional terms with “country” are used throughout this discussion, they apply as well to all participating states and other education 
systems. 
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rT Unlike TOTWGT, SENWGT, and HOUWGT, which are designed for use in student- 
level analyses from all student-level files, SCHWGT is designed for use in school- 
level analyses where the schools are the units of analysis. 


- TCHWGT, MATWGT, SCIWGT, and PHYWGT are specifically designed for 
analyses that link teacher background data to student data, for TIMSS and 
TIMSS Advanced 2015. 


- TCHWGT is used for analyses using all teachers for TIMSS 2015 grades 4 and 
8. 


- MATWGT and SCIWGT are used for analyses of mathematics and science 
teachers, respectively, for TIMSS 2015 grades 4 and 8. 


- MATWGT and PHYWGT are used for analyses of advanced mathematics and 
physics teachers, respectively, for TIMSS Advanced 2015. 


5.6 Structure and Design Variables in TIMSS and TIMSS 
Advanced 2015 Data Files 


The TIMSS and TIMSS Advanced 2015 data files also contain unique numerical identification variables 
for each respondent along with sample design information. 


5.6.1 Identification Variables 


In all TIMSS and TIMSS Advanced 2015 data files, identification variables are included to label 
countries, students, teachers, or schools. These variables also are used to link cases between the different 
data file types. The identification variables are the same across TIMSS and TIMSS Advanced 2015, have 
the prefix “ID,” and are described below. 


Ey IDCNTRY is a six-digit country identification code based on the ISO 3166 
classification. 


a IDPOP identifies the target grade; “1” for the fourth grade, “2” for the eighth grade, 
and “3” for TIMSS Advanced. 


. IDGRADE identifies the target grade of the participating students; “4” and “8” for 
most TIMSS countries and “12” for TIMSS Advanced. 


F IDSCHOOL is a four-digit identification code that uniquely identifies the 
participating schools within each country but are not unique across countries. 


. IDCLASS is a six-digit identification code that uniquely identifies the sampled 
classrooms within a country. 


a IDSTUD is an eight-digit identification code that uniquely identifies each sampled 
student in a country. 
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5 IDBOOK identifies the specific assessment booklet that was administered to each 
student. 


2 IDTEACH is a six-digit identification code that uniquely identifies a teacher within a 
school. 


f IDLINK uniquely identifies the class for which a teacher answered a questionnaire. 


5.6.2 Tracking Variables 


Information about students, teachers, and schools provided by the survey tracking forms described earlier 
is stored in the tracking variables. These variables have the prefix “IT.” All tracking variables are 
included in the student background data files. ITLANG is included in the student achievement and student 
background data files. 


5 ITSEX" is the gender of each student as stated in the Student Tracking Form. 


o ITBIRTHM and ITBIRTHY are respectively the month and year of birth of each 
student as stated in the Student Tracking Forms. 


7 ITDATE is the testing date for each student. 


7 ITLANG is the language of testing for each student. 


5.7 TIMSS and TIMSS Advanced 2015 Codebook Files 


Each data file released by NCES is accompanied by a codebook file. The codebooks are in the html 
format. The codebook files for public-use data can be downloaded from 
https://nces.ed.gov/timss/datafiles.asp. The codebook files for restricted-use data will require users to 
complete a restricted-use license agreement with NCES. Directions on how to obtain the license can be 


found at http://nces.ed.gov/pubsearch/licenses.asp. 


5.8 TIMSS and TIMSS Advanced 2015 U.S. Instrumentation 


As noted earlier, the U.S. national instrumentation differs from the international instrumentation in 
several ways: 


a Minor language/expression adaptations were made to some of the instructions on the 
school, teacher, and student questionnaires. 


fy Minor language adaptations were made to the wording of some assessment items. 


15% addition to the ITSEX variable that indicates student gender, the TIMSS and TIMSS Advanced data files also include student-reported 
gender variable (i.e., ASBG01 for TIMSS grade 4; BSBGO1 for TIMSS grade 8; for TIMSS Advanced, MSBG01 for Advanced Mathematics and 
PSBG01 for Physics) collected from the student questionnaire. 
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5 For a few questionnaire items, response alternatives were changed but in a way that 
allowed a crosswalk to the international response alternatives. 


7 Several U.S.-specific variables without international counterparts were added to the 
student and school questionnaires. 


Otherwise, the U.S. instrumentation is the same as the international instrumentation. For standard 
international versions of TIMSS and TIMSS Advanced 2015 questionnaires, please see Supplement 1 of 
either TIMSS or TIMSS Advanced 2015 User Guide for the International Database (links in the paragraph 
below). TIMSS and TIMSS Advanced 2015 also provide parallel supplements to the International User 
Guides. The instrumentation for the U.S. national and Florida samples are identical because Florida was 
administered the U.S. national instruments. Note that Florida participated in TIMSS 2015, but did not 
participate in TIMSS Advanced as a state. 


For TIMSS and TIMSS Advanced 2015, the U.S. versions of the Context Questionnaires can be 
downloaded from the NCES website at https://nces.ed.gov/timss/questionnaire.asp? Y ear=2015. 


For TIMSS, the supplements to the international user guide are as follows: 


7 Supplement 1, International Version of the TIMSS 2015 Context Questionnaires, can 
be downloaded from the TIMSS International Study Center at 
http://timssandpirls.bc.edu/timss2015/international- 


database/downloads/T15 UG Supplement1.pdf. 


a Supplement 2, the TIMSS National Adaptations of International Context 
Questionnaires of the TIMSS 2015 User Guide for the International Database, can be 
downloaded from the TIMSS International Study Center at 


http://timssandpirls.bc.edu/timss2015/international- 
database/downloads/T15 UG Supplement2.pdf. 


7 Supplement 3, the TIMSS Variables Derived from the Student, Home, Teacher, and 
School Questionnaire Data of the TIMSS 2015 User Guide for the International 
Database, can be downloaded from the TIMSS International Study Center at 
http://timssandpirls.bc.edu/timss2015/international- 


database/downloads/T15 UG Supplement3.pdf. 


7 The U.S. versions of the TIMSS questionnaires can be found at 
http://nces.ed.gov/timss/questionnaire.asp as well as in appendix D of the U.S. TIMSS 
and TIMSS Advanced 1995 & 2015 Technical Report and User’s Guide. The variable 
names in the U.S. files are identical to those in the international files with the 
exception of the U.S.-specific variables added. 


The TIMSS Advanced supplements to the TIMSS Advanced international user guide are as follows: 


a Supplement 1, International Version of the TIMSS Advanced 2015 Context 
Questionnaire, can be downloaded from the TIMSS Advanced International Study 


Center at http://timssandpirls.bc.edu/timss2015/advanced-international- 
database/downloads/TA15_UG_Supplement1.pdf. 
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5 Supplement 2, the TIMSS Advanced National Adaptations of International Context 
Questionnaires of the TIMSS Advanced 2015 User Guide for the International 
Database, can be downloaded from the TIMSS Advanced International Study Center 


at http://timssandpirls.bc.edu/timss2015/advanced-international- 
database/downloads/TA15_UG_Supplement2.pdf. 


7 Supplement 3, the TIMSS Advanced Variables Derived from the Student, Teacher, 
and School Questionnaire Data of the TIMSS Advanced 2015 User Guide for the 
International Database, can be downloaded from the TIMSS Advanced International 
Study Center at http://timssandpirls.bc.edu/timss2015/advanced-international- 
database/downloads/TA15 UG Supplement3.pdf. 


Py The TIMSS Advanced questionnaires may be downloaded from 
https://nces.ed.gov/timss/questionnaire.asp. They are also included in appendix D of 
the U.S. TIMSS and TIMSS Advanced 1995 & 2015 Technical Report and User’s 
Guide. The variable names in the U.S. files are identical to those in the international 
files with the exception of the U.S.-specific variables added. 


5.8.1 Background Questionnaire Items With U.S. Adaptations to Response 
Alternatives 


As the description of U.S. national adaptations in appendix E makes clear, there were a number of 
relatively minor changes to the wording of the international item stems and response alternatives in the 
questionnaires. Most of these adaptations do not require comment, as they are identical in format between 
the international and U.S. versions of the questionnaires (for example, they contain simple wording 
changes). In some cases, however, the adaptations resulted in item response formats not immediately 
comparable between the international and national versions of the questionnaires. As indicated in 
appendix E, there are instances in which the international and U.S. versions of variables have different 
sets of response codes; “instructional time” and “highest level of formal education” in the teacher 
questionnaire are two examples. This means that, for these items, the data are not identical in international 
and U.S. versions of the data files. For the “instructional time” item, the U.S. international version 
recorded total instructional time in minutes only whereas the national version recorded total instructional 
time in both hours and minutes. There were a few instances where the data in the national variables were 
changed to be consistent with the international version of the variables. For the “highest level of formal 
education” item, the variable has six response categories in the U.S. international file and seven categories 
in the U.S. national file. However, as indicated in appendix E, crosswalks between international and U.S. 
versions of these questions allow for the conversion of the U.S. response codes to the international 

format. 


5.8.2 U.S.-Specific Variables 


There are two types of U.S.-specific variables: (1) those with data from a source other than a 
questionnaire, and (2) those with data collected through questionnaires from national questions added to 
the international version of the questionnaires. 
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Variables with data from sources other than a questionnaire 
Data collected from school records covered the following: 


1. Whether a school was public or private. Information for this variable is derived from 
the Common Core Data (CCD). Although information about the type of school is 
available in the school questionnaire with nine separate categories that differentiate 
the types of public, private, charter, and alternative schools, the derived variable for 
public or private school provides a collapsed binary indicator for identifying schools 
as either public or private. Because the 9-category “type of school” variable came 
from the principals’ responses while the binary variable is derived from CCD, the data 
from the former variable may not match the data from the latter one. For example, for 
the schools with missing data in the 9-category type of school” variable, data are 
available in the derived variable. This variable is included in the public-use school 
files. 


2. Student disability status (SD). Information for this variable is derived from student 
lists submitted by the school coordinators of participating schools. This variable is in 
the restricted-use student files. 


3. Student English language learner status (ELL). Information for this variable is derived 
from student lists submitted by the school coordinators of participating schools. This 
variable is in the restricted-use student files. 


Coupled with school records, data collected from the College Board provided AP status for the U.S. 
TIMSS Advanced 2015 students. Two variables are derived based on the course type and AP course taken 
status (uncollapsed and collapsed). Uncollapsed variable include both AP and IB course categories. This 
variable provides the highest AP course taken for students who took AP courses. For non-AP students, 
the variable identifies if they took an IB course (either standard-level or high-level) or another advanced 
mathematics or physics course. In creating these derived variables, the primary source of data was the 
school records, which listed the courses taken by the students. If the school records were unclear or 
missing, then the College Board data collected for the students was checked to see which (if any) AP tests 
were taken. If there were no College Board data, any additional information collected from the school was 
used when determining their eligibility to participate in TIMSS Advanced 2015. Collapsed variable only 
includes AP and non-AP course categories. These variables are in the public-use student files. 


Data collected through the scoring process provided information found in the U.S.-specific diagnostic 
codes for TIMSS Advanced. The TIMSS Advanced 2015 scoring guides are described in TIMSS 2015 
Item Writing Guidelines (Mullis and Martin 2013). There were 12 U.S.-specific diagnostic codes for 
advanced mathematics constructed-response items and 8 for physics. Variables with the U.S.-specific 
diagnostic codes are included in the public-use student achievement files. Exhibits 5-8 and 5-9 provide 
the lists of international achievement items with corresponding national variables for advanced 
mathematics and physics that include U.S.-specific diagnostic codes. 
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Exhibit 5-8. 


TIMSS Advanced 2015 advanced mathematics achievement items with U.S.-specific 


diagnostic codes 


International Corresponding national 
variable name variable name 
MA33142 MA33142_N 
MA33178 MA33178_N 
MA33042 MA33042_N 
MA33091 MA33091_N 
MA23154 MA23154_N 
MA33012 MA33012_N 
MA33218 MA33218_N 
MA33039 MA33039_N 
MA23187 MA23187_N 
MA23043 MA23043_N 
MA33239 MA33239_N 
MA33243 MA33243_N 


U.S.-specific 


diagnostic codes 


77 


Explanation of diagnostic codes 


Common error 
Partial understanding 
Partial understanding 
Common error 
Partial understanding 
Partial understanding 
Partial understanding; common error 
Partial understanding 
Partial understanding 
Partial understanding 
Partial understanding 
Partial understanding 


SOURCE: U.S. Department of Education, National Center for Education Statistics, Trends in International Mathematics and Science Study 


(TIMSS), 2015. 


Exhibit 5-9. TIMSS Advanced 2015 physics achievement items with U.S.-specific diagnostic codes 


International Corresponding national 
variable name variable name 
PA33061B PA33061B_N 
PA23146 PA23146_N 

PA33058 PA33058_N 

PA33119 PA33119_N 

PA33046 PA33046_N 
PA33009A PA33009A_N 
PA33064A PA33064A_N 
PA33064B PA33064B_N 


U.S.-specific 


diagnostic codes 


78 
78 
77 
77,78 
77,78 
77,78 
77,78 
77,78 


Explanation of diagnostic codes 


Misconception 

Misconception 

Partial understanding 

Partial understanding; common error 
Partial understanding; common error 
Partial understanding; misconception 
Partial understanding; common error 
Partial understanding; misconception 


SOURCE: U.S. Department of Education, National Center for Education Statistics, Trends in International Mathematics and Science Study 


(TIMSS), 2015. 


Variables with data collected through questionnaires 


U.S.-specific items were added to the student, teacher, and school questionnaires. Nine questions were 
added to the TIMSS and TIMSS Advanced student questionnaires: 


1. 


2. 
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(TIMSS grades 4 and 8); 


a two-part question designed to measure the student’s race/ethnicity; 
a question that asked for language other than English spoken at home; 
a question about additional activities outside of school (TIMSS grades 4 and 8); 


a question about participation in science club, science fair, or science competition 


a question that asked about number of days absent; 
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6. a multiple-part question that asked students to indicate whether they had repeated a 
grade in school (TIMSS grades 4 and 8/TIMSS Advanced student questionnaires). 


7. a question that asked students to indicate how hard the test was compared to other 
tests they have taken this year in school; 


8. a question that asked students to indicate how hard did the students try on the test 
compared to other tests they have taken this year in school; and 


9. a question that asked students to indicate the importance of doing well on the test. 
Specifically for the TIMSS Advanced, six questions were added to the student questionnaires: 


1. a multiple-part question that asked students to indicate grade level and courses 
completed during high school for mathematics and physics courses; 


2. a multiple-part question that asked students to indicate Advanced Placement (AP) 
coursetaking during high school; 


3. a question that asked students to indicate online coursetaking for mathematics or 
science courses; 


4. a question that asked students to indicate International Baccalaureate (IB) 
coursetaking for mathematics or physics courses; 


5. a question that asked students to indicate activities related to postsecondary 
preparation that students took part during the school year; and 


6. a question that asked students to indicate extracurricular activities students 
participated during the school year. 


One question was added to the TIMSS and TIMSS Advanced teacher questionnaires that asked teachers 
to identify the year they started teaching. 


Specifically for the TIMSS grade 8 teacher questionnaires, one question was added to ask teachers to 
identify the title of the mathematics or science course being taught to the students being assessed. 


Three questions were added to the TIMSS and TIMSS Advanced school questionnaires: 
1. the percentage of students in the school eligible for free or reduced-price lunch; 
2. the percentage of students in the school who are English language learners; and 
3. a specification of the type of school. 


Specifically for the TIMSS grades 4 and 8, two additional questions were added to the school 
questionnaires: 


1. a question about the average income level of the school’s immediate area; and 


2. a multiple-part question about evaluating the practice of teachers. 
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For TIMSS Advanced, one additional question was added to the school questionnaire that asked 
principals whether their schools have a special program or track to prepare students for courses such as 
calculus or advanced physics. 


The following sections include detailed descriptions of the U.S.-specific variables. 


5.8.3 Race/Ethnicity (TIMSS 4 and 8/TIMSS Advanced Student Questionnaire) 


Students’ race/ethnicity was obtained through student responses to a two-part question in the student 
questionnaire. Students were asked first whether they were Hispanic or Latino and then whether they 
were members of the following five racial groups: (1) White; (2) Black or African American; (3) Asian; 
(4) American Indian or Alaska Native; or (5) Native Hawaiian or other Pacific Islander. Multiple 
responses to the second of these questions were allowed.!° A composite variable with six categories was 
constructed in which results are shown separately for (1) Whites; (2) Blacks; (3) Hispanics of any race; 
(4) Asians; and (5) Multiracial. The sixth category was labeled as “Other” and consisted of the small 
numbers of students indicating that they were American Indian or Alaska Native, and Native Hawaiian or 
other Pacific Islander. '” 


5.8.4 Language Other Than English Spoken at Home (TIMSS 4 and 8/TIMSS 
Advanced Student Questionnaire) 


This item extended the international question about how often students spoke English at home to ask 
those students who indicated that they did not always speak English if they spoke Spanish or another 
language. "8 


5.8.5 Additional Outside Activities (TIMSS 4 and 8 Student Questionnaire) 


The measure of outside of school activities was collected using a prompt with four yes/no questions. The 
prompt states, “The following questions ask about the activities you do outside of school. The yes/no 
questions were as follows: 


5 Do you play on a sports team outside of school? 


7 Do you play a musical instrument outside of school? 


6Race/ethnicity is provided with all categories in the restricted-use dataset. 

’Race/ethnicity is provided as a composite variable in the public-use dataset. The state benchmarking restricted-use datasets contain both the 
categorical and composite forms of the variable. 

18At grade 4 and 8, students who responded to items ASBGO3 (grade 4) and BSBG03 (grade 8) that they “always” speak English at home were 
supposed to skip items ASXGO03B (grade 4) and BSXGO03B (grade 8), which ask students to identify the foreign language that they sometimes 
use at home. However, about 12 percent of students (1246 cases) at grade 4, and 5 percent of students (464 cases) at grade 8 responded that they 
“always” speak English at home and also identified a foreign language that they sometimes use at home. For TIMSS Advanced, less than 4 
percent of advanced mathematics students and less than 3 percent of physics students who responded that they “always” speak English at home 
also identified a foreign language that they sometimes use at home. 
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5 Are you studying something in a class outside of school? 


a Do you belong to a club outside of school (like Girl Scouts, Cub Scouts, 4-H, or Boys 
and Girls Club)? 


5.8.6 Participation in Science Club, Science Fair, or Science Competition 
(TIMSS 4 and 8 Student Questionnaire) 


For TIMSS grade 4, students were asked if they are preparing for or have participated in a science club, a 
science fair, or a science competition in a yes/no question. For TIMSS grade 8, students were given three 
yes/no questions. The prompt states, “In this school year, are you preparing for or have you participated in 
any of the following activities?” The yes/no questions were as follows: 


| Science fair; 
| Science club; and 
a Science competition. 


5.8.7 Number of Days Absent (TIMSS 4 and 8/TIMSS Advanced Student 
Questionnaire) 


TIMSS and TIMSS Advanced students were asked about their school attendance over the last month prior 
to the assessment. The students were asked to select one response option. The question and response 
options were as follows: 
How many days were you absent from school in the last month? 

rT None; 

0 1 or 2 days; 

: 3 or 4 days; 

: 5 to 10 days; or 


im More than 10 days. 


5.8.8 Repeating a Grade (TIMSS 4 and 8/TIMSS Advanced Student 
Questionnaire) 


Eighth-grade students were asked whether they had ever repeated a grade in either “elementary school” 
and/or “middle or junior high school.” The response alternatives were yes/no in each case. Students who 
took TIMSS Advanced were asked the same question, but included an option for “high school.” 
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5.8.9 Difficulty of the Test (TIMSS 4 and 8/TIMSS Advanced Student 
Questionnaire) 


TIMSS Advanced students were asked how hard was the TIMSS Advanced assessment compared to most 
other tests they have taken this year in school. The response options were as follows: 


| Easier than other tests; 
| About as hard as other tests; 
rT Harder than other tests; or 


7 Much harder than other tests. 
5.8.10 Effort on the Test (TIMSS 4 and 8/TIMSS Advanced Student 
Questionnaire) 


TIMSS Advanced students were asked how hard they tried on the TIMSS Advanced assessment 
compared to most other tests they have taken this year in school. The response options were as follows: 


| Not as hard as on other tests; 
rT About as hard as other tests; 
a Harder than other tests; or 


7 Much harder than other tests. 
5.8.11 Importance of the Test (TIMSS 4 and 8/TIMSS Advanced Student 
Questionnaire) 


TIMSS Advanced students were asked how important it was to them to do well on the TIMSS Advanced 
assessment. The response options were as follows: 


7 Not very important; 
a Somewhat important; 
. Important; or 

a Very important. 
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5.8.12 Grade Level and Courses Completed During High School (TIMSS 


Advanced Student Questionnaire) 


TIMSS Advanced students were asked to indicate when and what mathematics or sciences courses they 


have completed in high school. The students were asked to indicate in what grade did they completed 
each of the courses listed in the question: (1) Never; (2) grade 8 or earlier; (3) grade 9; (4) grade 10; 
(5) grade 11; or (6) grade 12). Following is the list of courses for advanced mathematics student 


questionnaire: 


Algebra I course; 

Geometry course; 

Algebra II course, with or without trigonometry; 
Trigonometry (as a separate course); 

Pre-calculus course (also called introductory analysis); 
Calculus course; 

Probability or statistics course; 

Integrated mathematics 1 (first year of a multi-year course); 
Integrated mathematics 2 (second year of a multi-year course); 
Integrated mathematics 3 (third year of a multi-year course); 
Integrated mathematics 4 (fourth year of a multi-year course); and 


Other advanced mathematics course. 


Following is the list of courses for physics student questionnaire: 


General or unified science; 

Earth and space science; 

Life science (other than biology); 

Physical science (other than chemistry or physics); 
First-year biology; 

Second-year biology; 

First-year chemistry; 


Second-year chemistry; 
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First-year physics; 
Second-year physics; 
Engineering and technology; and 


Other advanced science course. 


5.8.13 Advanced Placement Coursetaking During High School (TIMSS 
Advanced Student Questionnaire) 


TIMSS Advanced students were asked to indicate if they have taken or are currently enrolled in any 
Advanced Placement (AP) courses. Following is the list of courses for the advanced mathematics 


student questionnaire: 


Advanced Placement (AP) Calculus AB; 
Advanced Placement (AP) Calculus BC; and 


Advanced Placement (AP) Statistics. 


Following is the list of courses for physics student questionnaire: 


Advanced Placement (AP) Biology; 

Advanced Placement (AP) Environmental Science; 
Advanced Placement (AP) Chemistry; 

Advanced Placement (AP) Physics B or C; !° and 


Advanced Placement (AP) Computer Science A or AB. 


5.8.14 Online Coursetaking (TIMSS Advanced Student Questionnaire) 


TIMSS Advanced students were asked whether they are currently enrolled or have taken online 
mathematics or science courses. The response options were as follows: 


No; 


Yes, for high school credit; 


19dvanced Placement (AP) Physics B was discontinued in 2014 by the College Board and replaced by AP Physics 1 and 2 for the 2015 exam 
administration. However, these changes in course titles are not reflected in the TIMSS Advanced 2015 student background questionnaire due to 
the timing of the study instrument preparations and data collection. 
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| Yes, for college credit; or 


rT Yes, for both high school and college credit. 


5.8.15 International Baccalaureate (IB) Coursetaking (TIMSS Advanced 
Student Questionnaire) 


TIMSS Advanced students were asked whether they are currently enrolled or have taken an International 
Baccalaureate (IB) mathematics or science courses. The response options were yes/no. 


5.8.16 Activities Related to Postsecondary Preparation (TIMSS Advanced 
Student Questionnaire) 


TIMSS Advanced students were asked to indicate postsecondary education preparation-related activities 
that students have done during the current school year. Following is the list of activities: 


. Taken the SAT or ACT college entrance exams; 

4 Submitted the Free Application for Federal Student Aid (FAFSA); 
a Applied to a 2-year college; 

Ey Been accepted to a 2-year college; 


rT Applied to a 4-year college; 


a Been accepted to a 4-year college; 
7 Talked with a military recruiter or contacted a ROTC program; 
a Enlisted in the military or enrolled in a ROTC program; 


om Applied for a full-time job; 
7 Been interviewed for a full-time job; and 


7 None of the above. 
5.8.17 Extracurricular Activities (TIMSS Advanced Student Questionnaire) 


TIMSS Advanced students were asked to indicate whether they have participated in various 
extracurricular activities during the current school year. Following is the list of extracurricular activities: 


7 Sports; 


. Performing arts; 
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Academic clubs; 
Vocational/professional clubs; 
Honor societies; 

Publications; 

Student government; 

Service clubs; and 


Hobby clubs. 


5.8.18 Year Started Teaching (TIMSS 4 and 8/TIMSS Advanced Teacher 
Questionnaire) 


TIMSS and TIMSS Advanced teachers were asked to indicate the year that they started teaching. 


5.8.19 Mathematics Course Taught (TIMSS 8 Teacher Questionnaire) 


Eighth-grade mathematics teachers were asked about the nature of the mathematics course they taught to 
the TIMSS students. The response alternatives provided were as follows: 


Basic or general eighth-grade mathematics (not algebra or pre-algebra); 
Introduction to algebra or pre-algebra; 

Two-year pre-algebra; 

Algebra I (1-year course); 

Algebra I (first year of a 2-year algebra I course); 

Algebra I (second year of a 2-year algebra I course); 

Geometry; 

Algebra II; 

Integrated or sequential mathematics; and 


Other mathematics class. 
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5.8.20 Science Course Taught (TIMSS 8 Teacher Questionnaire) 


Eighth-grade science teachers were asked about the nature of the science course they taught to the TIMSS 
students. The response alternatives provided were as follows: 


7 General science (several content areas of science taught separately); 


a Integrated science (several content areas of science taught combined and taught 
throughout the year); 


7 Life science (e.g., biology, ecosystems, human health); 
7 Physical science (e.g., physics or chemistry); and 
m Earth science (e.g., geology, earth and the solar system, fossils). 


5.8.21 Poverty Level in Public Schools (Percentage of Students Eligible for 
Free or Reduced-Price Lunch) (TIMSS 4 and 8/TIMSS Advanced School 
Questionnaire) 


The measure of poverty level in public schools was obtained from principals’ responses to the school 
questionnaire. The question asked the principal to report, as of approximately October 2014, the 
percentage of students at the school eligible to receive free or reduced-price lunch through the National 
School Lunch Program (NSLP). This continuous variable is provided in the restricted-use school file. To 
provide data for public-use, Common Core Data (CCD) are used to derive a new variable with five 
categories: (1) less than 10 percent, (2) 10 to 24.9 percent, (3) 25 to 49.9 percent, (4) 50 to 74.9 percent, 
and (5) 75 percent or more.”° Because the continuous form of the variable came from the principals’ 
responses while the categorical form of the variable is derived from CCD, the data from the former form 
may not match the data from the latter form. For example, if data from the school questionnaire is 
unavailable, information from the CCD is recorded in the public-use school file, but recorded as 
“missing” in the restricted-use school file. The effect of this replacement on the confidentiality of the data 
was examined as part of the confidentiality analyses described earlier in chapter 4 in Section 4.6.2. 


5.8.22 Limited-English Proficient (LEP)/English Language Learners (ELL) 
(TIMSS 4 and 8/TIMSS Advanced School Questionnaire) 


Principals were asked to report the percentage of such students and were provided with the following 
eight response categories: (1) 0 percent; (2) 1-5 percent; (3) 6-10 percent; (4) 11—25 percent; (5) 26-50 
percent; (6) 51-75 percent; (7) 76-90 percent; and (8) over 90 percent. 


20 Florida restricted-use datasets contain both the categorical and composite forms of the variable. 
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5.8.23 Type of School (TIMSS 4 and 8/TIMSS Advanced School Questionnaire) 


Principals were asked to identify their schools using one of the following 10 response categories: 

(1) regular public school; (2) regular public school with magnet program; (3) magnet school or school 
with special program; (4) special education; (5) alternative curriculum; (6) vocational; (7) charter school; 
(8) independent private school; (9) religiously affiliated private school; or (10) other school. 


5.8.24 Average Income Level of School’s Immediate Area (TIMSS 4 and 8 
School Questionnaire) 


Principals were asked to indicate the average income level of the school’s immediate area using one of 
the following three response categories: (1) high; (2) medium; or (3) low. 


5.8.25 Evaluating the Practice of Teachers (TIMSS 4 and 8 School 
Questionnaire) 


Principals were asked to indicate practices that were used to evaluate teachers at their schools in yes/no 
responses. Following is the list of practices for evaluating teachers: 


7 Observations by the principal or senior staff; 

cy Observations by inspectors or other persons external to the school; 
P Student achievement; and 

= Teacher peer review. 


5.8.26 Special Program for Calculus or Advanced Physics (TIMSS Advanced 
School Questionnaire) 


Principals were asked to indicate whether they have a special program or track to prepare students for 
courses such as calculus or advanced physics in yes/no response. 
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5.8.27 Missing Data 


Data derived from the student, school and teacher questionnaires and from the student assessments 
contain missing data in varying amounts. Four sources of missing data are identified: 


1. Not administered. The respondent was not administered the actual item. He or she had 
no chance to read and answer the question. 


2. Omitted or invalid. The respondent had a chance to answer the question but did not do 
so. This code also was used for responses that were not interpretable. 


3. Logically not applicable. The respondent answered a preceding filter question in a 
way that made the following dependent questions not applicable to him or her. 


4. Not reached (only used in the achievement files). This code indicates those items not 
reached by the students due to a lack of time. 


SAS and SPSS control code for all the data files include the code for handling/converting missing data. 
The control code files are programs that contain data definition statements used by SAS and SPSS 
statistical software to generate data files from flat data files in .dat format. They provide list of statements 
that specify the name, type, decimal specification, and location for each variable in the data files. The 
control code files also provide statements that assign variables labels (i.e., variable descriptions), missing 
values, and value labels based on category descriptions. 


5.8.28 Imputation 


No imputation for missing values was undertaken. However, missing data on the measure of school 
poverty (proportion of students eligible for free or reduced-price lunch) reported by schools was replaced 
as described in the Poverty Level in Public Schools section of this chapter. 


5.9 TIMSS Advanced 1995 U.S. National Data Files 


As noted earlier, in addition to the TIMSS Advanced 2015 data files, a set of the TIMSS Advanced 1995 
school and student data files are provided in public-use and restricted-use data formats for trend analysis 
purposes. No teacher questionnaire was administered to the teachers of students in their final year of 
secondary school for TIMSS Advanced 1995; therefore, no teacher data files are available for TIMSS 
Advanced 1995. 


Similar to the TIMSS Advanced 2015 U.S. national data files, the TIMSS Advanced 1995 U.S. national 
instrumentation differs from the international instrumentation in several ways: 


5 Minor language/expression adaptations were made to some of the instructions. 


an Minor language adaptations were made to the wording of some assessment items. 
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5 For a few questionnaire items, response alternatives were changed but in a way that 
allowed a crosswalk to the international response alternatives. 


7 For a few questionnaire items, U.S.-specific adaptations with international 
counterparts were added to replace the international questionnaire item structure. 


Additional U.S.-specific variables without international counterparts were added to the student and school 
questionnaires. Otherwise, the TIMSS Advanced 1995 U.S. instrumentation is the same as the 
international instrumentation. For the standard international version of the TIMSS Advanced 1995 
questionnaires, see Supplement 1 of the User Guide for the TIMSS International Database: Final Year of 
Secondary School (link provided in the paragraph below). Other supplements to the International User 
Guides are described in the paragraph below. 


2 Supplement 1, International Versions of the Background Questionnaires — Population 
3, can be downloaded from the TIMSS International Study Center at 


https://timssandpirls.bc.edu/timss1995i/database/UG3_Sup1.pdf. 


. Supplement 2, Documentation of National Adaptations of International Background 
Questionnaire Items — Population 3, can be downloaded from the TIMSS International 
Study Center at https://timssandpirls.bc.edu/timss1995i/database/UG3_Sup2.pdf. 


7 Supplement 3, Variables Derived from the Student Questionnaire — Population 3, can 
be downloaded from the TIMSS International Study Center at 
https://timssandpirls.bc.edu/timss1995i/database/UG3_Sup3.pdf. 


A The U.S. versions of the TIMSS Advanced 1995 school and student questionnaires 
can be found at https://nces.ed.gov/timss/questionnaire.asp? Y ear=1995. The variable 
names in the U.S. files are identical to those in the international files with the 
exception of the U.S.-specific variables added. 


5.9.1 TIMSS Advanced 1995 U.S. National Adapted Variables 


In TIMSS Advanced 1995, U.S.-specific national adaptations were added to replace the international 
counterpart questionnaire items in the school questionnaires. These national adaptations were created to 
replace the international questionnaire structure in order to capture U.S.-specific information. 


Two school questionnaire items had national adaptations added to replace the international questionnaire 
structure: 


1. a multipart question that asked principals to indicate whether mathematics and science 
courses are available to Grade 12 students in the school; and 


2. a multipart question that asked principals to indicate graduation requirements. 
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5.9.2 Courses Available to Grade 12 Students in School (TIMSS Advanced 


1995 School Questionnaire) 


In TIMSS Advanced 1995, principals were asked to indicate yes or no on whether a list of mathematics 
and science courses are available to grade 12 students in their school. Following is the list of courses in 
the school questionnaire: 


General mathematics; 
Business/consumer math; 
Algebra I; 

Algebra II; 

Geometry; 

Analytic geometry; 
Trigonometry; 

Statistics; 

Math analysis; 
Pre-calculus; 

Calculus; 

AP calculus; 

General science; 

Earth science; 

Physical science; 

Biology; 

Biology, 2nd year (including botany, zoology, etc.); 
Human health and nutrition; 
Human anatomy and/or physiology 
Environmental science; 
Chemistry I; 


Chemistry II; 
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AP chemistry; 
Astronomy; 
Conceptual physics; 
Physics I; 

Physics II; 

AP physics; and/or 


Other science course. 


5.9.3 Graduation Requirements (TIMSS Advanced 1995 School 
Questionnaire) 


In TIMSS Advanced 1995, principals were asked to indicate graduation requirements in their schools. 
Following is the list of graduation requirement in the school questionnaire: 


Pass a comprehensive (multi-subject) examination; 

Pass an examination in mathematics; 

Pass an examination in science; 

Maintain a certain standard of performance (e.g., grade point average); 
Accumulate a certain amount of credit/course work; 

Attend a specified number of years; 

Reach a certain age; and/or 


Pass an oral examination. 


5.9.4 TIMSS Advanced 1995 U.S.-Specific Variables 


As with TIMSS Advanced 2015, there were two types of U.S.-specific variables in 1995: (1) those with 
data from a source other than a questionnaire, and (2) those with data collected through questionnaires 
from national questions added to the international version of the questionnaires. 
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Variables with data from sources other than a questionnaire 


1. Information about whether a school is public or private is derived from the Common 
Core Data (CCD). This variable is in the public-use school files and serves as an 
indicator of whether a school is public or private. 


2. A variable named COMPARISON that flags students in the 1995 sample who can be 
compared with the 2015 sample (for details, see below under section 5.13.7, Special 
Considerations in Comparing the TIMSS Advanced 1995 and 2015 Student Data). 


Variables with data collected through questionnaires 
In TIMSS Advanced 1995, U.S.-specific items were added to the school and student questionnaires. One 


additional question was added to the school questionnaire that asked principals about expectations for 
parental involvement at school. Five additional questions were added to the student questionnaires: 


1. a question that asked about student’s race/ethnicity; 

2. a three-part question that asked students about studying mathematics before or after 
school; 

3. a three-part question that asked students about studying science before or after school; 

4. a question about time student spent each day on mathematics homework; and 

5. a question about time student spent each week on science homework. 


5.9.5 Race/Ethnicity (TIMSS Advanced 1995 Student Questionnaire) 


Students’ race/ethnicity was obtained through student responses to a question in the student questionnaire. 
Students were asked to identify which one of the following five racial/ethnic groups best describes them: 
(1) White (not Hispanic); (2) Black (not Hispanic); (3) Hispanic; (4) Asian or Pacific Islander; or 

(5) American Indian or Alaska Native. Alternatively, they could also choose “Other (specify)” as one of 
the choices. Multiple responses to the question choices were not allowed. A derived uncollapsed variable 
with six categories was constructed in which results are shown separately for (1) White; (2) Black; 

(3) Hispanic; (4) Asian; and (5) American Indian. The sixth category was labeled as “Other” and 
consisted of the small numbers of students indicating that they were either multiracial or had selected 
“Other” from the response choice. Student race/ethnicity is provided with uncollapsed categories in the 
restricted-use dataset. For the public-use dataset, student race/ethnicity is provided as a collapsed 
composite variable with (1) White; (2) Black; (3) Hispanic; (4) Asian; and (5) Other. 


5.9.6 Studying Mathematics/Science Before or After School (TIMSS 
Advanced 1995 Student Questionnaire) 


TIMSS Advanced students were asked about studying mathematics and science before or after school. 
The students were asked to indicate “yes” or “no” for where they studied, with whom they studied, and 
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specific educational-related home possessions for studying. The yes/no questions for each part were as 
follows: 


Think of the last time you studied mathematics/science before or after school. Where did you 


study? 
a At your home; 
a At a friend’s home; 


: At school (for example, in study hall); 


a In a special class paid for by your parents; 

. Ina local library; 

7 On the way to or from school (for example, on the bus or train); and/or 
7 Other. 


Think of the last time you studied mathematics/science before or after school. With whom did 
you study? 


rT By myself; 


| With a friend; 


| With several friends; 
| With a class; and/or 
a With a special teacher (for example, a tutor). 


Think of the last time you studied mathematics/science before or after school. Did you have? 
a A quiet area/room in which to work; 
7 Your own reference books, workbooks, etc.; and/or 


7 Your own study desk/table; 


5.9.7 Daily Time Spent on Mathematics Homework (TIMSS Advanced 1995 
Student Questionnaire) 


TIMSS Advanced students were asked about how much time they spent each day on mathematics 
homework. The response options were (1) I am not taking math this year; (2) None; (3) 15 minutes; 
(4) 30 minutes; (5) 45 minutes; (6) 1 hour; or (7) More than 1 hour. 
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5.9.8 Weekly Time Spent on Science Homework (TIMSS Advanced 1995 
Student Questionnaire) 


TIMSS Advanced students were asked about how much time they spent each week on doing science 
homework. The response options were: (1) I am not taking science this year; (2) None; (3) 30 minutes; 
(4) 1 hour; (5) 2 hours; (6) 3 hours; or (7) More than 3 hours. 


5.9.9 Missing Data 


Similar to TIMSS and TIMSS Advanced 2015, TIMSS Advanced 1995 data derived from the school and 
student questionnaires and from the student assessments contain missing data in varying amounts. Four 
sources of missing data are identified: 


1. Not administered. The respondent was not administered the actual item. He or she had 
no chance to read and answer the question. 


2. Omitted or invalid. The respondent had a chance to answer the question but did not do 
so. This code also was used for responses that were not interpretable. 


3. Logically not applicable. The respondent answered a preceding filter question in a way 
that made the following dependent questions not applicable to him or her. 


4. Not reached (only used in the achievement files). This code indicates those items not 
reached by the students due to a lack of time. 


5.9.10 Imputation 


No imputation for missing values was undertaken for the TIMSS Advanced 1995 data files. 


5.10 Merging Data Files 


In preparing TIMSS 2015 or TIMSS Advanced 2015 data for analysis it may be necessary to merge two 
(or more) of the data files named in exhibits 5-1 and 5-2. Not every analysis requires merging of files, 
however. For example, analyses looking at the relationship between student background and achievement 
can be done using the student background file alone. However, analyses that examine the relationships 
between school and/or teacher and/or student characteristics and student achievement require that files be 
merged. Standard merging procedures as implemented in SPSS, SAS or Stata can be applied. Examples 
are provided below along with illustrative SAS and SPSS code. (These various merges are facilitated for 
SPSS and SAS users who choose to work with the IEA International Database [IDB] described below.) 


The merging procedures illustrated below follow the same pattern as previous TIMSS studies (see Foy 
2017a for TIMSS Grades 4 and 8; see Foy 2017b for TIMSS Advanced) and are illustrated with TIMSS 
grade 8 data. The user needs to replace file names in order to run the merges for TIMSS grade 4 or 
TIMSS Advanced. 
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5.10.1 Merging Student and School Data 


If the intent is to disaggregate school data across students, the school-level data are merged to the student 
file using IDSCHOOL. The disaggregated data can be analyzed at the student level using the student-level 
weight TOTWGT. Exhibits 5-10 and 5-11 provide examples of how to merge the student and school data 
using SAS and SPSS. Additional examples are provided in chapters 2 and 3 in the TIMSS 2015 User 
Guide for the International Database (Foy 2017a) and TIMSS Advanced 2015 User Guide for the 
International Database (Foy 2017b)). 


Exhibit 5-10. Illustrative SAS code for merging U.S. TIMSS eighth-grade student and school data 


libname bM6 “C:\TIMSS2015\Data”; 


data SCHOOL; 
set bM6.T8_SCHOOL15; 


proc sort data= SCHOOL; 
by IDSCHOOL; 


data STUDENT; 
set bM6.T8_STUDENT15; 


proc sort data= STUDENT; 
by IDSCHOOL; 


data bM6.MERGF1; 

merge STUDENT SCHOOL; 
by IDSCHOOL; 

run; 


een ern eRe eee eee. a a a ae ee 
SOURCE: International Association for the Evaluation of Education Achievement (IEA), Trends in International Mathematics and Science Study 
(TIMSS), 2015. 


This example creates a temporary SAS dataset (SCHOOL) using the permanent school dataset 
T8_SCHOOL15 located in the directory defined as bME. It then sorts the school data by school ID 
(IDSCHOOL). A similar procedure is used to create the temporary student file (STUDENT), which is 
also sorted by the school ID using the permanent student dataset T8_STUDENT15 in the bM6 directory. 
The final dataset is a permanent dataset to be stored in the bM6 directory called MERGE1, which 
contains the merged file from SCHOOL and STUDENT using IDSCHOOL as the merge variable. 


The SPSS example shown in exhibit 5-11 works in a similar way. SPSS uses a file containing the school 

variables (T8_SCHOOL15.SAV) and sorts the cases by IDSCHOOL. The same procedure is used for the 
student dataset, T8_STUDENT15.SAV. The “match files” command merges the two files, and the final, 

merged output file is saved as MERGE1.SAV. 
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Exhibit 5-11. Illustrative SPSS code for merging U.S. TIMSS eighth-grade student and school data 


get file = 'C:\TIMSS2015\Data\T8_SCHOOL15.SAV'. 
sort cases by IDSCHOOL. 
save outfile = "C:\TIMSS2015\Data\SCHOOL.SAV". 


get file = 'C:\TIMSS2015\Data\T8_STUDENT15.SAV'. 
sort cases by IDSCHOOL. 
save outfile = "C:\TIMSS2015\Data\STUDENT.SAV". 


match files 

/ file= "C:\TIMSS2015\Data\STUDENT.SAV" 

/ table= "C:\TIMSS2015\Data\SCHOOL.SAV" 

/ by IDSCHOOL. 

save outfile = 'C:\TIMSS2015\Data\MERGE1.SAV'. 


SOURCE: International Association for the Evaluation of Education Achievement (IEA), Trends in International Mathematics and Science Study 
(TIMSS), 2015. 


5.10.2 Merging Student and Teacher Data 


In the United States, the TIMSS grades 4 and 8 student samples were based on intact classrooms. The 
teachers of the students selected in this way are not a sample of teachers and should be seen as the 
“teachers of the sampled students.” To maintain this linkage, merges of teacher and student data must use 
the student-teacher link file (T8_STD_TCH_LINK15.DAT), which also contains the appropriate teacher 
sampling weights. Exhibits 5-12 and 5-13 provide illustrative code for merging the student and the 
mathematics teacher files in SAS and SPSS. The sample code provided illustrates the merge involving the 
eighth-grade mathematics teacher file with the student file. The same logic applies to a merge involving 
the science teacher file (T8_STEACHER15.DAT). Additional examples of how to merge the student and 
teacher files using SAS and SPSS are provided in chapters 2 and 3 of the TIMSS 2015 User Guide for the 
International Database ((Foy 2017a) and TIMSS Advanced 2015 User Guide for the International 
Database (Foy 2017b)). 
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Exhibit 5-12. Illustrative SAS code for merging U.S. TIMSS eighth-grade student and mathematics 
teacher data 


libname bM6 “C:\TIMSS2015\Data”; 


data TEACHER; 
set bM6.T8_MTEACHERIS5; 


proc sort data= TEACHER; 
by IDTEACH IDLINK; 


data STDTCH; 
set bM6.T8_STD_TCH_LINK15; 


proc sort data= STDTCH; 
by IDTEACH IDLINK; 


data TEACHMRG; 

merge TEACHER STDTCH; 
by IDTEACH IDLINK; 

if MATWGT > 0; 


proc sort data = TEACHMRG; 
by IDSTUD; 


data STUDENT; 
set bM6.T8_STUDENT15; 


proc sort data = STUDENT; 
by IDSTUD; 


data bM6.MERGE2; 

merge STUDENT TEACHMRG; 
by IDSTUD; 

run; 


1 ae Sn ee eS a a RE ee eS ee he a i 
SOURCE: International Association for the Evaluation of Education Achievement (IEA), Trends in International Mathematics and Science Study 
(TIMSS), 2015. 


In the SAS example, the program creates a temporary SAS dataset (TEACHER) using the permanent 
mathematics teacher file, T8_MTEACHER15, located in the directory defined as bME. It then sorts the 
teacher data by the teacher ID (IDTEACH) and the link ID (IDLINK). A similar procedure is used for the 
student-teacher link file (STDTCH), using the permanent file T8_STD_TCH_LINK15 in the bM6 
directory, which is also sorted by the teacher ID and the link ID. The weight variable for mathematics 
teachers (MATWGT) is used as a selection variable because mathematics teachers have been selected. 
The result is a merged file called TEACHMRG with disaggregated teacher data. This file is merged with 
the student file (STUDENT). The final dataset is a permanent dataset called MERGE2 in the bM6 
directory that contains the merged file from TEACHMRG and STUDENT using IDSTUD as the merge 
variable. 


The SPSS student-teacher merge in exhibit 5-13 uses a file containing the teacher variables 
(T8_MTEACHER15.SAV) and sorts the cases by IDTEACH and IDLINK. The file is then saved as 
TEACHER. The same procedure is used for the student-teacher linkage dataset 
T8_STD_TCH_LINK15.SAV. The “match files” command merges the two files by the ID variables 
IDTEACH and IDLINK, and the merged output file is saved as TEACHMRG. To include the student 
data, the student file is selected (T8_STUDENT15.SAV), sorted by IDSTUD and saved as STUDENT. 
This file is merged with TEACHMRG using IDSTUD to create the final file MERGE2.SAV containing 
both teacher and student variables. 
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Exhibit 5-13. Illustrative SPSS code for merging U.S. TIMSS eighth-grade student and mathematics 
teacher data 


get file = 'C:\TIMSS2015\Data\T8_MTEACHER15.SAV'. 
sort cases by IDTEACH IDLINK . 
save outfile = "C:\TIMSS2015\Data\TEACHER.SAV". 


get file = 'C:\TIMSS2015\Data\T8_STD_TCH_LINK15.SAV'. 
select if MATWGT > 0. 

sort cases by IDTEACH IDLINK. 

save outfile= "C:\TIMSS2015\Data\STDTCH.SAV". 


match files 

/ file="C:\TIMSS2015\Data\STDTCH.SAV" 

/ table="C:\TIMSS2015\Data\TEACHER.SAV" 

/ by IDTEACH IDLINK. 

sort cases by IDSTUD. 

save outfile = "C:\TIMSS2015\Data\TEACHMRG.SAV". 


get file = "C:\TIMSS2015\Data\T8_STUDENT15.SAV". 
sort cases by IDSTUD. 
save outfile= "C:\TIMSS2015\Data\STUDENT.SAV". 


match files 

/ file= ""C:\TIMSS2015\Data\TEACHMRG.SAV" 

/ table= "C:\TIMSS2015\Data\STUDENT.SAV" 

/ by IDSTUD. 

save outfile = "C:\TIMSS2015\Data\MERGE2.SAV". 


Ral ae ee ee oe ee Ee A SE a SE A SS NS A EN A 
SOURCE: International Association for the Evaluation of Education Achievement (IEA), Trends in International Mathematics and Science Study 
(TIMSS), 2015. 


5.10.3 Merging Student, School, and Teacher Data 


In merging student, teacher, and school data together to form a single dataset, the procedures from 
sections 5.10.1 and 5.10.2 are combined. Exhibits 5-14 and 5-15 show illustrative SAS and SPSS code 
designed to achieve this three-way merge. Note that the teacher weight variables are specifically designed 
for using teacher background data in student-level analyses and are located in the student-teacher linkage 
files not in the teacher background data files. Analyses with teacher data need to be weighted properly by 
merging the teacher files with the student-teacher linkage files. As in the previous examples, the sample 
code is based on eighth-grade mathematics teacher file. A merge involving the eighth-grade science 
teacher file follows the same logic. 


This example uses the same merging steps as with the previous school and teacher examples (MERGE1 


and MERGE2), and then merges the output files by the student ID, IDSTUD, into a final permanent file 
MERGEALL containing linked student, school, and teacher data at the student level. 
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Exhibit 5-14. Illustrative SAS code for merging U.S. TIMSS eighth-grade school, mathematics teacher, 
and student data 


libname bM6 “C:\TIMSS2015\Data”; 


data SCHOOL; 
set bM6.T8_SCHOOL15; 


proc sort data= SCHOOL; 
by IDSCHOOL; 


data STUDENT; 
set bM6.T8_STUDENT15; 


proc sort data= STUDENT; 
by IDSCHOOL; 


data MERGE1; 
merge STUDENT SCHOOL; 
by IDSCHOOL; 


proc sort data=MERGE1; 
by IDSTUD; 


data TEACHER; 
set bM6.T8_MTEACHERI15; 


proc sort data= TEACHER; 
by IDTEACH IDLINK; 


data STDTCH; 
set bM6.T8_STD_TCH_LINK15; 


proc sort data= STDTCH; 
by IDTEACH IDLINK; 


data MERGE2; 

merge STDTCH TEACHER; 
by IDTEACH IDLINK; 

if MATWGT > 0; 

proc sort data = MERGE2; 
by IDSTUD; 


data bM6.MERGEALL; 
merge MERGE1 MERGE2; 
by IDSTUD; 

run; 


a i I 
SOURCE: International Association for the Evaluation of Education Achievement (IEA), Trends in International Mathematics and Science Study 
(TIMSS), 2015. 


In the SPSS example shown in exhibit 5-15 the student and school data are first sorted by IDSCHOOL 
and then merged. The procedure followed for combining student and teacher data in exhibit 5-13 is used 
again. Then the saved student-school and student-teacher files are merged by IDSTUD, and a final dataset 
MERGEALL.SAV is saved. 
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Exhibit 5-15. Illustrative SPSS code for merging U.S. TIMSS eighth-grade school, mathematics 
teacher, and student data 


get file = 'C:\TIMSS2015\Data\T8_SCHOOL15.SAV'. 
sort cases by IDSCHOOL. 
save outfile = "C:\TIMSS2015\Data\SCHOOL.SAV". 


get file = 'C:\TIMSS2015\Data\T8_STUDENT15.SAV'. 
sort cases by IDSCHOOL. 
save outfile = "C:\TIMSS2015\Data\STUDENT.SAV". 


match files 

/file= "C:\TIMSS2015\Data\STUDENT.SAV" 
/table= '"C:\TIMSS2015\Data\SCHOOL.SAV" 

/by IDSCHOOL. 

sort cases by IDSTUD(A). 

save outfile = "C:\TIMSS2015\Data\MERGE1.SAV". 


get file = 'C:\TIMSS2015\Data\T8_MTEACHER15.SAV'. 
sort cases by IDTEACH IDLINK. 
save outfile = "C:\TIMSS2015\Data\TEACHER.SAV". 


get file = 'C:\TIMSS2015\Data\T8_STD_TCH_LINK15.SAV'. 
select if MATWGT > 0. 

sort cases by IDTEACH IDLINK. 

save outfile = "C:\TIMSS2015\Data\STDTCH.SAV". 


match files 

/file= "C:\TIMSS2015\Data\STDTCH.SAV" 

/table= '"C:\TIMSS2015\Data\TEACHER.SAV" 

/by IDTEACH IDLINK. 

sort cases by IDSTUD(A). 

save outfile = "C:\TIMSS2015\Data\MERGE2.SAV". 


match files 

/file= "C:\TIMSS2015\Data\MERGE2.SAV" 

/table= "C:\TIMSS2015\Data\MERGE1.SAV" 

/by IDSTUD. 

save outfile = "C:\TIMSS2015\Data\MERGEALL.SAV". 


SOURCE: International Association for the Evaluation of Education Achievement (IEA), Trends in International Mathematics and Science Study 
(TIMSS), 2015. 


5.10.4 Merging TIMSS or TIMSS Advanced 2015 Data with Restricted-Use Data 


Users who have been granted a license to use the restricted-use TIMSS or TIMSS Advanced 2015 data 
will receive a restricted-use CD-ROM that contains an additional link file. This file provides a way to 
merge TIMSS or TIMSS Advanced data with school data from the Common Core of Data (CCD) and the 
Private School Survey (PSS). The NCESSCH (the NCES unique public school identification code) from 
the TIMSS or TIMSS Advanced file is used to merge with NCESSCH from the CCD file. The PPIN (the 
private school’s unique identification number) from the TIMSS or TIMSS Advanced file is used to merge 
with the PPIN from the PSS file. Illustrative SAS and SPSS code is provided in exhibits 5-16 and 5-17. 
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The SAS code in question provides for a link between the TIMSS school data and the CCD/PSS data by 
school. Further merging to other TIMSS files (student, school, and teacher) can be conducted using the 
IDSCHOOL for merging as has been shown in earlier examples. 


Exhibit 5-16. Illustrative SAS code for merging U.S. TIMSS eighth-grade school data with restricted- 
use (CCD and PSS) data 


libname bM6 “C:\TIMSS2015\Data”; 


data SCHOOL; 
set bM6.T8_RESTRICTED_USE15; 


proc sort data= SCHOOL; 
by NCESSCH; 


data CCD; 
set bM6.CCD; 


proc sort data= CCD; 
by NCESSCH; 


data MERGE1; 

merge CCD(IN=IN1) SCHOOL (IN=IN2); 
by NCESSCH; 

IF IN2; 

run; 


/* User can Merge in PSS data to the file containing CCD and TIMSS previously merged data */ 


data SCHOOL2; 
set MERGF1; 


proc sort data= SCHOOL2; 
by PPIN; 


data PSS; 
set bM6.PSS; 


proc sort data= PSS; 
by PPIN; 


data MERGE2; 

merge PSS(IN=IN1) SCHOOL2 (IN=IN2); 
by PPIN; 

IF IN2; 

run; 


NN Pil 
SOURCE: International Association for the Evaluation of Education Achievement (IEA), Trends in International Mathematics and Science Study 
(TIMSS), 2015. 


SPSS code designed to provide this same link between the TIMSS school data and the CCD/PSS data by 
school is provided in exhibit 5-17. Further merging to other TIMSS files (student, school, and teacher) 
can be conducted using the IDSCHOOL for merging as has been shown in earlier examples. 
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Exhibit 5-17. Illustrative SPSS code for merging U.S. TIMSS eighth-grade school data with restricted 
(CCD and PSS) data 


get file = 'C:\TIMSS2015\Data\T8_RESTRICTED_USE15_SCH.SAV'. 
sort cases by NCESSCH. 
save outfile = "C:\TIMSS2015\Data\SCHOOL.SAV". 


get file = 'C:\TIMSS2015\Data\CCD.SAV'. 
sort cases by NCESSCH. 
save outfile = "C:\TIMSS2015\Data\CCD.SAV". 


match files 

/file= "C:\TIMSS2015\Data\CCD.SAV" 

/table= '"C:\TIMSS2015\Data\SCHOOL.SAV" 

/by NCESSCH. 

save outfile = "C:\TIMSS2015\Data\MERGE1.SAV". 


/* Merge PSS to the Combined TIMSS/CCD school-level file.*/ 


get file = 'C:\TIMSS2015\Data\MERGE1.SAV'. 
sort cases by PPIN. 
save outfile = "C:\TIMSS2015\Data\SCHOOL2.SAV". 


get file = 'C:\TIMSS2015\Data\PSS.SAV'. 
sort cases by PPIN. 
save outfile = "C:\TIMSS2015\Data\PSS.SAV". 


match files 

/file= "C:\TIMSS2015\Data\PSS.SAV" 

/table= "C:\TIMSS2015\Data\SCHOOL2.SAV" 

/by PPIN. 

save outfile = "C:\TIMSS2015\Data\MERGE2.SAV". 


SOURCE: International Association for the Evaluation of Education Achievement (IEA), Trends in International Mathematics and Science Study 
(TIMSS), 2015. 


5.11 Some Notes on Analyzing the TIMSS and TIMSS Advanced 
2015 Data 


The design of TIMSS and TIMSS Advanced raises several special considerations for the analysis of 
TIMSS and TIMSS Advanced data. First, the assessment design necessitates the use of five plausible 
values rather than a single score for each of the various measures of mathematics and science for grades 4 
and 8, and advanced mathematics and physics for TIMSS Advanced. Second, since the sampling design is 
not a simple random sample in which each student had an equal probability of selection, sampling weights 
need to be applied to generate unbiased estimates of population parameters. Third, the complex sampling 
design also means that the calculation of the standard errors of the various statistics generated requires 
special procedures. 
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5.11.1 Plausible Values 


As noted earlier, the assessment design was based on Balanced Incomplete Block (BIB) spiraling of 
assessment items to increase content-area coverage without a concomitant increase in the assessment time 
demanded of students. Each student completed only a subset of the total pool of assessment items, with 
the resulting data containing missing values for other items in the pool but not in the subset administered 
to the student. The trade-off for increased coverage through BIB spiraling is increased measurement error 
in the scores available for each student. This is accommodated through the estimation of five plausible 
values for each student rather than a single unreliable point estimate. Plausible values are random draws 
from the estimated distribution of a student’s achievement. A detailed description of the TIMSS or 
TIMSS Advanced 2015 scaling can be found in Methods and Procedures in TIMSS 2015 (Martin, Mullis, 
and Hooper 2016a) and Methods and Procedures in TIMSS Advanced 2015 (Martin, Mullis, and Hooper 
2016b). 


What this means for those analyses of TIMSS and TIMSS Advanced data that include achievement 
measures is that the analyses need to be done five times and the results averaged. For example, if one was 
regressing mathematics achievement on a number of family and school attributes, it would be necessary 
to estimate this equation five times and then average each set of five parameter estimates. It would not be 
legitimate to take the mean of the five plausible values in the first instance and then regress this mean on a 
number of family and school attributes. 


5.11.2 Estimating Sampling Variance 


The complex sampling design used in TIMSS and TIMSS Advanced 2015 complicates the task of 
computing standard errors. Most standard analysis software systems such as SAS and SPSS provide 
estimates based on the assumption of a simple random sample. Given the TIMSS and TIMSS Advanced 
sampling design, such standard errors underestimate the true standard errors. TIMSS and TIMSS 
Advanced adopt the jackknife repeated replication (JRR) technique because it is computationally 
straightforward and provides approximately unbiased estimates of the sampling errors of means, totals, 
and percentages. The variables necessary for these JRR procedures are included as part of the TIMSS and 
TIMSS Advanced 2015 data files: JKZONE, the sampling zone (stratum) of the student’s school; and 
JKREP, the sampling replicate (primary sampling unit) of the student’s school. 


There are, however, several options for estimating sampling errors that avoid the assumption of simple 
random sampling. The SPSS- and SAS-linked International Database (IDB) Analyzer software is 
designed specifically by International Association for the Evaluation of Educational Achievement (IEA) 
for analyzing TIMSS and TIMSS Advanced international data files. This software is freely available from 
the IEA website at http://www.iea.nl/data.html. 


Special-use software is also available for estimating the standard errors of statistics generated from 
complex sampling designs. Among the packages available are AM, available from the American Institutes 
for Research at http://am.air.org/about2.asp; WesVar, available from Westat at 
https://www.westat.com/our-work/information-systems/wesvar-support/download-wesvar; and SUDAAN, 
available from Research Triangle Institute at http://www.rti.org/sudaan. Some software packages provide 
for these capabilities as well. 
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In addition, SAS macros suitable for this purpose are available as part of the TIMSS 2015 User Guide for 
the International Database, as well as in the TIMSS Advanced 2015 User Guide for the International 
Database (Foy 2017a; Foy 2017b). See also the work by Stapleton (2006, 2008), which suggests 
procedures that can be used to generate appropriate standard errors for statistics generated by structural 
equation modeling techniques. 


5.11.3 The IEA International Database (IDB) Analyzer and International Data 
Explorer (IDE) 


As described in section 5.3.3 and 5.11.2, the IEA IDB Analyzer was developed by the IEA Data 
Processing and Research Center (IEA DPC) as a plug-in for SPSS and SAS. It can be used in conjunction 
with SPSS and SAS. It is not a stand-alone analysis system. 


The IEA IDB Analyzer enables users to combine SPSS and SAS data files and conduct analyses using 
SPSS and SAS without actually writing programming code. The IEA IDB Analyzer generates SPSS 
syntax and SAS program that take into account information from the sampling design in the computation 
of statistics and their standard errors. In addition, the generated SPSS syntax and SAS program makes 
appropriate use of plausible values for calculating estimates of achievement scores and their standard 
errors, combining both sampling variance and imputation variance. 


The IEA IDB Analyzer consists of two modules—a merge module and an analysis module. The merge 
module is used to create analysis datasets by combining data files of different types and from different 
countries and selecting subsets of variables for analysis. The analysis module provides procedures for 
computing various statistics and their standard errors. All statistical procedures offered within the analysis 
module of the IEA IDB Analyzer make appropriate use of sampling weights, and standard errors are 
computed using the jackknife repeated replication (JRR) method. Percentages, means, regressions, and 
correlations may be specified with or without achievement scores. When achievement scores are used, the 
analyses are performed five times—once for each plausible value—and the results are aggregated to 
produce accurate estimates of achievement and standard errors that incorporate both sampling and 
imputation errors. 


The use of the IEA IDB Analyzer is described in detail with worked examples in chapter 2 of both the 
TIMSS 2015 User Guide for the International Database and the TIMSS Advanced 2015 User Guide for 
the International Database (Foy 2017a; Foy 2017b). Readers intending to use this user-friendly software 
are urged to read this user guide in detail. Researchers who intend to use the IEA IDB Analyzer with 
SPSS should execute the SPSS programs. Likewise, those who intend to use the IEA IDB Analyzer with 
SAS should execute the SAS programs described in the chapter 3 of the TIMSS 2015 User Guide for the 
International Database and the TIMSS Advanced 2015 User Guide for the International Database (Foy 
2017a; Foy 2017b). 


In addition to IEA IDB Analyzer for basic analysis and exploration of TIMSS and TIMSS Advanced data, 
NCES has developed a relatively simple, interactive online data analysis tool: the International Data 
Explorer (IDE), which can be found at http://nces.ed.gov/surveys/international/ide. The IDE allows users 
to analyze all the international variables for all participating education systems and the U.S.-specific 
variables; however, it does not include U.S. restricted-use data. The IDE does not require SPSS or SAS 
for analyzing the data. It provides users with the capabilities to create statistical tables and charts of 
TIMSS and TIMSS Advanced data across countries and years on the website. This tool allows users to 
point and click in a self-contained module, unlike the IEA IDB Analyzer software that must be used in 
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conjunction with SPSS and SAS. Also unlike the IEA IDB Analyzer, the IDE does not provide access to 
data files for merging, transforming, or otherwise manipulating data. This tool reports averages for 
subject by selected variables and exports reports in HTML, Excel, Word, or PDF. 


5.11.4 SAS Programs and Macros 


The TIMSS 2015 User Guide for the International Database and the TIMSS Advanced 2015 User Guide 
for the International Database (Foy 2017a; Foy 2017b) also provide assistance for those investigators 
who wish to conduct their analyses using SAS. The user guide includes a number of SAS programs 
needed to process the SAS data files, compute survey results, and carry out example analyses. These are 
described in detail with worked examples in chapter 3 of the user guide. Readers intending to use SAS for 
their analyses are urged to read this chapter in detail. 


The following SAS programs and macros are available at 
https://timssandpirls.bc.edu/timss2015/international-database/downloads/T15_Programs.zip (for TIMSS 
2015) and https://timssandpirls.bc.edu/timss2015/advanced-international- 

database/downloads/TA15 Programs.zip (for TIMSS Advanced 2015): 


rT T15_CONVERT.SAS and TA15_CONVERT.SAS are used to convert SAS Export 
files into SAS data files. Additional information about how to use the converting SAS 
programs is provided in chapter 3 of the TIMSS 2015 User Guide for the International 
Database and the TIMSS Advanced 2015 User Guide for the International Database 
(Foy 2017a; Foy 2017b). 


| ASASCRME.SAS, BSASCRM6.SAS, MSASCRM3.SAS, and PSASCRM3.SAS are 
used to convert the response codes on the achievement items to their corresponding 
score levels. 


5.11.5 Special Considerations in Using the Teacher Data 


The teachers in the TIMSS and TIMSS Advanced 2015 data files are the teachers of nationally 
representative samples of students and are not representative samples of teachers in the participating 
countries. As a result, analyses with teacher data should be made with students as the units of analysis and 
reported in terms of students who are taught by teachers with a particular attribute. 


When analyzing teacher data, it is first necessary to link the students to their respective teachers. The 
student-teacher linkage data files were created for this purpose. Since student achievement scores 
(plausible values), jackknife replication information, and teacher weighting variables are found in the 
student-teacher linkage data files, it is only necessary to merge the teacher background data files with the 
student-teacher linkage data files. For analyses linking teacher variables to student background variables, 
it is also necessary to merge the student background data files with the teacher background data files after 
having been combined with the student-teacher linkage data files. 
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In general, to perform analyses using the teacher background data files, follow the steps below. 


1. 


Identify the variables of interest in the teacher background data files and note any 
specific national adaptations to the variables. 


Retrieve the relevant variables from the teacher background data files, including analysis 
variables, classification variables, identification variables (IDCNTRY, IDTEACH, and 
IDLINK), and any other variables used in the selection of cases. 


Retrieve the relevant variables from the student-teacher linkage data files, including 
plausible values of achievement, classification variables, identification variables 
(IDCNTRY, IDSTUD, IDTEACH, and IDLINK), sampling (JKZONE and JKREP) and 
weighting (MATWGT, SCIWGT, or TCHWGT) variables, and any other variables used 
in the selection of cases. 


Merge the teacher background data files with the student-teacher linkage data files using 
the variables IDCNTRY, IDTEACH, and IDLINK. 


If student background variables also are needed, merge the student background data files 
with the merged student-teacher data files from the previous step using the variables 
IDCNTRY and IDSTUD. 


One further point to note: fourth-grade teachers were given a single questionnaire with mathematics and 
science sections. If a teacher taught only mathematics or only science to the TIMSS fourth-graders, that 
teacher would complete only one of these sections. In frequency distributions of the variables in these 
sections, teachers who did not answer the questions for this reason are shown as “not administered.” 


5.11.6 Special Considerations in Using the School Data 


In general, to perform analyses using the school background data files, follow the steps below. 


dl 


Identify the variables of interest in the school and student background data files and note 
any specific national adaptations to the variables. 


Retrieve the relevant variables from the school background data files, including analysis 
variables, classification variables, identification variables IDCNTRY and IDSCHOOL), 
and any other variables used in the selection of cases. 


Retrieve the relevant variables from the student background data files, including 
plausible values of achievement, classification variables, identification variables 
(IDCNTRY and IDSCHOOL), sampling (JKZONE and JKREP) and weighting 
(TOTWGT) variables, and any other variables used in the selection of cases. 


Merge the school background data files with the student background data files using the 
variables IDCNTRY and IDSCHOOL. 
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5.11.7 Special Considerations in Comparing the TIMSS Advanced 1995 and 
2015 Student Data 


As noted previously, in addition to the TIMSS Advanced 2015 data files, a set of the TIMSS Advanced 
1995 school and student data files are provided in public-use and restricted-use data formats for trend 
analysis purposes. The 1995 student files contain a variable, COMPARISON, which serves as an 
indicator variable if the student can be used for analyses that compare TIMSS Advanced 1995 and 2015 
students. A value of one (1) indicates that the student is included in the comparison group, while a value 
of two (2) indicates the student is not included in the comparison group. 


As described in chapter 2 of this technical report, the TIMSS Advanced student population is defined 
based on a student’s mathematics and science coursetaking during high school. The target population for 
advanced mathematics is defined as students in the final year of secondary schooling who have taken 
courses in advanced mathematics. The target population for physics is defined as students in the final year 
of secondary schooling who have taken courses in physics. The courses that define the target populations 
have to cover most, if not all, of the advanced mathematics and physics topics that are outlined in the 
TIMSS Advanced 2015 Assessment Frameworks (Mullis and Martin 2014). In addition, chapter 2 of this 
technical report provides sampling information and summarized description of the IEA TIMSS Advanced 
framework for both advanced mathematics and physics. 


For the purpose of TIMSS Advanced 2015, participation eligibility for advanced mathematics is defined 
as students who have taken or were taking a calculus course at the time of the assessment, and for physics 
is defined as students who have taken or were taking a second-year physics course. To ensure eligibility 
to participate in the study, the actual courses taken by students are provided by schools and confirmed by 
both school staff and school course catalogs. 


The TIMSS Advanced 2015 target population differs from that for TIMSS Advanced 1995, for which 
students were designated as “specialists” in advanced mathematics and/or physics by the school. An 
advanced mathematics “specialist” is defined as a student who has taken a pre-calculus, calculus, or AP 
calculus course. A physics “specialist” is defined as a student who has taken a first-years physics course 
(which did not include physical science courses, but may have included general physics courses). Student 
who met criteria for advanced mathematics were eligible to take the advanced mathematics assessment. 
Student who met criteria for physics were eligible to take the physics assessment. Students who met both 
criteria of advanced mathematics and physics were considered as “combined specialists” and were 
eligible to take a combined advanced mathematics-and-physics assessment. Students who met neither set 
of criteria were defined as “generalists” (or “non-specialists”) and were eligible to take the twelfth-grade 
mathematics and science literacy assessment. 


Although the trend analysis is conducted on comparable students based on their coursework, the manner 
in which the coursework has been identified and substantiated differs in the two administrations and 
should be recognized. The criteria used to identify a subset of the U.S. students in the TIMSS Advanced 
1995 population that is comparable with the 2015 sample and to generate the COMPARISON variables in 
the 1995 student data files are based on the following: 


4 school identifying the student as a specialist for advanced mathematics and/or physics; 
and 

F student self-reported coursetaking information from the student background 
questionnaire. 
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As part of the TIMSS Advanced 1995 student background questionnaire, students were asked to report 
the highest level course taken in mathematics and science during high school. The student responses from 
the questionnaire were used to differentiate students who took calculus and advanced physics courses. If a 
school defined a student as an advanced mathematics “specialist,” and the student reported taking a 
calculus course, then the student is included in the advanced mathematics subset that is comparable to the 
TIMSS Advanced 2015 advanced mathematics sample (i.e. COMPARISONS = 1 in the 
TA_M_STUDENT95 data file). Similarly, if a school defined a student as a physics “specialist,” and the 
student reported taking a second-year physics, Advanced Placement (AP), or International Baccalaureate 
(IB) physics course, then the student was included in the physics subset that is comparable to the TIMSS 
Advanced 2015 physics sample (i.e, COMPARISONS = 1 in the TA_P_STUDENT95 data file). For 
additional information about TIMSS Advanced 1995 databases, please see the User Guide for the TIMSS 
International Database—Final Year of Secondary School (Gonzalez, Smith, and Sibberns 1998). 
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A-1. TIMSS 2015 and TIMSS Advanced 2015 State Letter 
Red text should be customized before mail merge: highlighted text represents mail merge fields 


UNITED STATES DEPARTMENT OF EDUCATION 
INSTITUTE OF EDUCATION SCIENCES 


NATIONAL CENTER FOR EDUCATION STATISTICS 


[Date] 


Dear [Title] [Name Last]: 


The United States is participating in an important international study in 2015: the Trends in 
International Mathematics and Science Study (TIMSS). TIMSS is the longest ongoing 
international assessment of student achievement in mathematics and science. Since 1995, 
TIMSS has measured trends in academic achievement at grades 4 and 8 in countries around the 
world, including the United States. In 2015, TIMSS will also assess advanced mathematics and 
physics at grade 12. Known as TIMSS Advanced, this 12th-grade assessment will provide 
education policymakers with valuable information about how many high school students are 
excelling at highly specialized science, technology, engineering, and mathematics (STEM) 
content in a global context. Results from these assessments are used by researchers and 
policymakers to chart national progress against international standards and other countries 
around the world, informing national discussions about international competitiveness. 


Some schools in your state have been randomly selected to participate in TIMSS in spring 2015. 
I am writing to ask your agency to support the participation of those selected schools. 


TIMSS is described in more detail in the enclosed materials. In the United States, TIMSS is 
sponsored by the National Center for Education Statistics (NCES), part of the U.S. Department 
of Education, and is conducted by Westat, a research organization based in Rockville, Maryland. 
The U.S. Office of Management and Budget has approved the data collection under OMB 
#1850-0695. For information on the confidentiality of the data collected, please see the enclosed 
FAQ. While participation in this study is voluntary, your support of school participation in your 
state is invaluable so that the United States has a representative sample of schools across the 
country. 


Within the next few weeks, your NAEP State Coordinator will contact sampled school districts 
to inform them of their schools selection for the assessment. In the meantime, if you have 
questions about the study, please do not hesitate to call Dr. Chris Averett at 240-314-2492 or 
send an email to TIMSS@westat.com. You may also get more information about these studies 
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A-1. TIMSS 2015 and TIMSS Advanced 2015 State Letter—Continued 


by contacting Dr. Stephen Provasnik at NCES at 202-502-7480 or stephen.provasnik@ed.gov, or 
by visiting the TIMSS website at https://nces.ed.gov/TIMSS/. 


Thank you for your time and support. 
Sincerely, 
/Peggy Carr, Ph.D./ 


Peggy Carr, Ph.D. 

Associate Commissioner, Assessment Division 

National Center for Education Statistics 

cc: 

Enclosures 

NCES is authorized to conduct this study under the Education Sciences Reform Act of 2002 (ESRA 2002), 20 U.S. Code, § 9543. By lam, the data 


provided by your schools, staff, and students may only be used for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose 
except as required by law (20 U.S. Code, § 9573). 
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A-2. TIMSS 2015 and TIMSS Advanced 2015 District Letter 


Early May TIMSS/TIMSS Advanced Notification Letter From 
CHIEF STATE SCHOOL OFFICER TO DISTRICT SUPERINTENDENT 
Red text should be customized before mail merge: highlighted text represents mail merge fields 


Dear (District Superintendent name): 


lam writing to notify you that (number) schools in your district have been randomly selected to participate in the 
2015 administration of an important international study in 2015: the Trends in International Mathematics and 
Science Study (TIMSS). TIMSS is the longest ongoing international assessment of student achievement in 
mathematics and science. Since 1995, TIMSS has measured trends in academic achievement at grades 4 and 8 in 
countries around the world, including the United States. In 2015, TIMSS will also assess advanced mathematics 
and physics at grade 12. Known as TIMSS Advanced, this 12th-grade assessment will provide education 
policymakers with valuable information about how many high school students are excelling at highly specialized 
science, technology, engineering, and mathematics (STEM) content in a global context. Results from these 
assessments are used by researchers and policymakers to chart national progress against international standards 
and other countries around the world, informing national discussions about international competitiveness. 


Selected schools are notified in advance so that principals can place the assessment window on their calendars and 
incorporate TIMSS into the planned school program. | am writing to ask for your support of the participation of 
those selected schools. Each school will receive a confidential report with information about how its students 
performed on TIMSS 2015's released items (i.e., questions made public after the assessment). Information about 
how students performed across the United States and in other participating countries on those same released 
items will be made public, and schools can compare their students' performance on the released items against 
those national and international results (provided the school meets response rate and sample size requirements). 
Participating schools will receive $200, and each school’s TIMSS school coordinator (the school staff person 
designated to work with TIMSS representatives) will receive $100 as a thank you for his or her time and effort. 
Each student who participates will receive a small gift as a token of appreciation. 


TIMSS is described in more detail in the enclosed materials. In the United States, TIMSS is sponsored by the 
National Center for Education Statistics (NCES), part of the U.S. Department of Education, and is conducted by 
Westat, a research organization based in Rockville, Maryland. The U.S. Office of Management and Budget has 
approved the data collection under OMB #1850-0695. For information on the confidentiality of the data collected, 
please see the enclosed FAQ. While participation in this study is voluntary, your support of school participation in 
your district is invaluable so that the United States has a representative sample of schools across the country. The 
schools will be contacted soon with more information about the TIMSS assessment. 


The list of selected schools from your district is attached. Please include the TIMSS assessment window (March 30 
to May 29, 2015) on your district test calendar. (Name), our NAEP State Coordinator, will contact your staff with 
additional information. 


If you have any questions, please do not hesitate to contact me, or the toll-free TIMSS information hotline at 855- 
445-5604 or TIMSS@westat.com. You may also get more information about this study by contacting Dr. Stephen 
Provasnik at NCES at 202-502-7480 or stephen.provasnik@ed.gov, or by visiting the TIMSS website at 
http://nces.ed.gov/timss/. 


| know that | can count on you to help accomplish our goal of 100 percent participation. Thank you for your time 
and support. 
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A-2. TIMSS 2015 and TIMSS Advanced 2015 District Letter—Continued 
(Name of Chief State School Officer) 
Enclosures: TIMSS folder 
CC: State Testing Director 
District Test Director 
NAEP State Coordinator 
NCES is authorized to conduct this study under the Education Sciences Reform Act of 2002 (ESRA 2002), 20 U.S. 
Code, § 9543. By law, the data provided by your schools, staff, and students may only be used for statistical 


purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law 
(20 U.S. Code, § 9573). 
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A-3. TIMSS 2015 School Letter 


May TIMSS 2015 Notification Letter From 
NAEP STATE COORDINATOR TO SCHOOL PRINCIPAL — TIMSS 2015 — Grades 4 & 8 
Red text should be customized before mail merge: highlighted text represents mail merge fields 


Dear (Principal name): 


lam writing to inform you that (school name) has been selected to represent schools across the United States by 
participating in an important international study in 2015: the Trends in International Mathematics and Science 
Study (TIMSS). TIMSS is the longest ongoing international assessment of student achievement in mathematics and 
science. Since 1995, TIMSS has measured trends in academic achievement at grades 4 and 8 in countries around 
the world, including the United States. Results from these assessments are used by researchers and policymakers 
to chart national progress against international standards and other countries around the world, informing 
national discussions about international competitiveness. 


Your school has been randomly selected to participate in TIMSS in spring 2015. Your school will receive a 
confidential report with information about how your students performed on TIMSS 2015's released items (i.e., 
questions made public after the assessment). Information about how students performed across the United States 
and in other participating countries on those same released items will be made public, and you can compare your 
students’ performance on the released items against those national and international results. Participating schools 
will receive $200, and each school’s TIMSS school coordinator (the school staff person designated to work with 
TIMSS representatives) will receive $100 as a thank you for his or her time and effort. Participating students will 
receive a small gift as a token of appreciation. 


TIMSS is described in more detail in the enclosed materials. In the United States, TIMSS is sponsored by the 
National Center for Education Statistics (NCES), part of the U.S. Department of Education, and is conducted by 
Westat, a research organization based in Rockville, Maryland. The U.S. Office of Management and Budget has 
approved the data collection under OMB #1850-0695. For information on the confidentiality of the data collected, 
please see the enclosed FAQ. | hope you will participate in this voluntary study because it is important that the 
United States has a nationally representative sample of schools. 


For now, | am writing only to notify you of the assessment. The assessment window is March 30-May 29, 2015. In 
June, | will send you an assessment date. Should there be a conflict on that date, a TIMSS representative will work 
with you to identify an alternate. Our goal is to schedule the assessment date prior to the beginning of the school 
year so that you may include it on your 2014-2015 school calendar. At the beginning of the school year, | will send 
you detailed information about the assessment and will ask you to identify a school coordinator. TIMSS 
representatives will provide significant support to schools, bring all necessary materials, and administer the 
assessment. 


If you have any questions, please do not hesitate to contact me, or the toll-free TIMSS information hotline at 855- 
445-5604 or TIMSS @westat.com. You may also get more information about this study by contacting Dr. Stephen 
Provasnik at NCES at 202-502-7480 or stephen.provasnik@ed.gov, or by visiting the TIMSS website at 
http://nces.ed.gov/timss/. 


Your participation in the TIMSS 2015 is very important to its success. Our chief state school officer, (name), and 
your district superintendent, (name), support TIMSS and look forward to your school’s participation. Thank you for 
your time and for supporting this important international study. 


NAEP State Coordinator 
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A-3. TIMSS 2015 School Letter—Continued 

Enclosures: TIMSS folder 

cc: District Test Coordinator 

NCES is authorized to conduct this study under the Education Sciences Reform Act of 2002 (ESRA 2002), 20 U.S. 
Code, § 9543. By law, the data provided by your school, staff, and students may only be used for statistical 


purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law 
(20 U.S. Code, § 9573). 
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A-4. TIMSS Advanced 2015 School Letter 


May TIMSS 2015 Notification Letter From 
NAEP STATE COORDINATOR TO SCHOOL PRINCIPAL — TIMSS Advanced 2015 — Grade 12 
Red text should be customized before mail merge: highlighted text represents mail merge fields 


Dear (Principal name): 


lam writing to inform you that (school name) has been select to represent schools across the United States by 
participating in an important international study in 2015: the Trends in International Mathematics and Science 
Study (TIMSS). TIMSS is the longest ongoing international assessment of student achievement in mathematics and 
science. Since 1995, TIMSS has measured trends in academic achievement at grades 4 and 8 in countries around 
the world, including the United States. In 2015, TIMSS will also assess advanced mathematics and physics at grade 
12. Known as TIMSS Advanced, this 12th-grade assessment will provide education policymakers with valuable 
information about how many high school students are excelling at highly specialized science, technology, 
engineering, and mathematics (STEM) content in a global context. Results from these assessments are used by 
researchers and policymakers to chart national progress against international standards and other countries 
around the world, informing national discussions about international competitiveness. 


Your school has been randomly selected to participate in TIMSS Advanced in spring 2015. Your school will receive a 
confidential report with information about how your students performed on TIMSS 2015's released items (i.e., 
questions made public after the assessment). Information about how students performed across the United States 
and in other participating countries on those same released items will be made public, and you can compare your 
students' performance on the released items against those national and international results (provided your school 
meets response rate and sample size requirements). Participating schools will receive $200, and each school’s 
TIMSS school coordinator (the school staff person designated to work with TIMSS representatives) will receive 
$100 as a thank you for his or her time and effort. Participating students will receive a small gift as a token of 
appreciation. 


TIMSS Advanced is described in more detail in the enclosed materials. TIMSS Advanced is sponsored in the United 
States by the National Center for Education Statistics (NCES) in the U.S. Department of Education and is conducted 
by Westat, a research organization based in Rockville, Maryland. The U.S. Office of Management and Budget has 
approved the data collection under OMB #1850-0695. For information on the confidentiality of the data collected, 
please see the enclosed FAQ. | hope you will participate in this voluntary study because it is important that the 
United States has a nationally representative sample of schools. 


For now, | am writing only to notify you of the assessment. The assessment window is March 30-May 29, 2015. In 
June, | will send you an assessment date. Should there be a conflict on that date, a TIMSS representative will work 
with you to identify an alternate. Our goal is to schedule the assessment date prior to the beginning of the school 
year so that you may include it on your 2014-2015 school calendar. At the beginning of the school year, | will send 
you detailed information about the assessment and will ask you to identify a school coordinator. TIMSS 
representatives will provide significant support to schools, bring all necessary materials, and administer the 
assessment. 


If you have any questions, please do not hesitate to contact me, or the toll-free TIMSS information hotline at 855- 
445-5604 or TIMSS@westat.com. You may also get more information about this study by contacting Dr. Stephen 
Provasnik at NCES at 202-502-7480 or stephen.provasnik@ed.gov, or by visiting the TIMSS website at 


http://nces.ed.gov/timss/. 


Your participation in the TIMSS 2015 Advanced is very important to its success. Our chief state school officer, 
(name), and your district superintendent, (name), support TIMSS and look forward to your school’s participation. 
Thank you for your time and for supporting this important international study of preparedness for STEM careers. 
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A-4. TIMSS Advanced 2015 School Letter—Continued 

NAEP State Coordinator 

Enclosures: TIMSS folder 

cc: District Test Coordinator 

NCES is authorized to conduct this study under the Education Sciences Reform Act of 2002 (ESRA 2002), 20 U.S. 
Code, § 9543. By law, the data provided by your school, staff, and students may only be used for statistical 


purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law 
(20 U.S. Code, § 9573). 


U.S. TIMSS 2015 and TIMSS Advanced 1995 & 2015 Technical Report and User’s Guide A-9 


A-5. TIMSS 2015 Parent Notification Letter: Grade 4 


SCHOOL LETTERHEAD 


Dear Parent or Guardian, 


This letter is to inform you about an important international study of student learning being conducted 
in our school this spring. The Trends in International Mathematics and Science Study (TIMSS) provides 
important information for benchmarking student performance in mathematics and science at grades 4 
and 8 in the United States against countries around the world. Since 1995, TIMSS has measured 
worldwide trends in student knowledge of mathematics and science. The next TIMSS assessment will be 
in spring 2015. 


Our school has accepted an invitation from the National Center for Education Statistics (NCES), part of 
the U.S. Department of Education, to participate in TIMSS 2015. {Insert number} of our 4th-grade classes 
will take part. {This/One of these} is your child’s class. The enclosed summary sheet provides some 
background information about TIMSS, explains what is involved for each student selected to participate 
in the study, and gives a contact phone number and email address where you can find answers to any 
questions you might have. 


To have an accurate picture of what U.S. 4th-graders can do in mathematics and science, it is important 
that each student selected take part in the study. In addition to answering mathematics and science 
questions, students will be asked to complete a brief questionnaire about themselves. | urge you to 
support this effort by encouraging your child to take part; however, participation in this study is entirely 
voluntary. Previous experience suggests that students enjoy taking part, and participating students will 
receive a small gift that we think they will like. 


All of the information collected is safeguarded, as required by law. NCES is authorized to conduct this 
study under the Education Sciences Reform Act of 2002 (ESRA, 2002), 20 U.S. Code, Section 9543. Under 
that law, the data provided by schools, staff, and students may be used only for statistical purposes and 
may not be disclosed or used in identifiable form for any other purpose except as required by law (20 
U.S.C., § 9573). Students and schools are never identified in any reports. All reported statistics refer to 
the United States as a whole. 


Thank you for taking the time to learn about this important study. 
Sincerely, 
[School Official] 


Enclosure: 
Facts for Parents: TIMSS 2015 
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A-6. TIMSS 2015 Parent Notification Letter: Grade 4 (Spanish version) 


SCHOOL LETTERHEAD 


Estimado padre/madre o tutor, 


Por medio de la presente queremos informarle acerca de un importante estudio internacional que se 
realizara en nuestra escuela esta primavera acerca del aprendizaje de los estudiantes. El Estudio 
Internacional sobre las Tendencias en Matematicas y Ciencias (TIMSS, por sus siglas en inglés) 
proporciona informacion importante para evaluar el rendimiento de los estudiantes de 4° y 8° grado en 
Estados Unidos en matematicas y ciencias en comparacidn con otros paises del mundo. Desde 1995, 
TIMSS ha medido las tendencias a nivel mundial en el aprendizaje de los estudiantes de matematicas y 
ciencias. 


Nuestra escuela ha aceptado una invitacion del Centro Nacional de Estadisticas de la Educacion (NCES, 
por sus siglas en inglés), parte del Departamento de Educacidn de Estados Unidos, para participar en 
TIMSS 2015. Participaran {Insert number} de nuestras clases de 4° grado. {Esta/Una de estas} es la clase 
de su hijo. El resumen adjunto ofrece informacidn de trasfondo de TIMSS, explica lo que implica la 
participacion en el estudio para cada estudiante seleccionado e incluye un numero de teléfono y un 
correo electronico de contacto donde usted podra encontrar respuestas a cualquier pregunta que tenga. 


Para tener una imagen precisa de lo que los estudiantes de 4° grado en Estados Unidos pueden hacer en 
matematicas y ciencias, es importante que cada estudiante seleccionado participe en el estudio. Ademas 
de contestar las preguntas de matematicas y ciencias, a los estudiantes se les pedira que contesten un 
cuestionario corto acerca de ellos mismos. Quisiéramos pedirle que apoye esta iniciativa animando a su 
hijo a participar; sin embargo, la participacidn en este estudio es completamente voluntaria. Las 
experiencias anteriores sugieren que los estudiantes disfrutan de la participacién. Ademas, los 
estudiantes que participen recibiran un pequefio regalo que creemos les gustara. 


Toda la informaci6n que se retina esta protegida, como lo exige la ley. NCES esta autorizado a realizar 
este estudio de acuerdo con la Ley de Reforma de Ciencias de la Educaci6n de 2002 (ESRA, 2002), Cédigo 
de Estados Unidos, Titulo 20, Secci6n 9543. Segun esa ley, la informacién proporcionada por las 
escuelas, el personal y los estudiantes Unicamente se puede usar con fines estadisticos y no se puede 
dar a conocer ni usar de alguna manera que pueda identificar a los participantes para ningun otro fin, 
excepto cuando lo exige la ley (20 U.S.C., § 9573). En los informes nunca se identifica ni a los estudiantes 
nia las escuelas. Todas las estadisticas publicadas se refieren a Estados Unidos en conjunto. 


Gracias por tomarse el tiempo de informarse acerca de este importante estudio. 
Atentamente, 
[School Official] 


Documento anexo: 
Informacion para los padres: TIMSS 2015 
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A-7. TIMSS 2015 Parent Notification Letter: Grade 8 


SCHOOL LETTERHEAD 


Dear Parent or Guardian, 


This letter is to inform you about an important international study of student learning being conducted 
in our school this spring. The Trends in International Mathematics and Science Study (TIMSS) provides 
important information for benchmarking student performance in mathematics and science at grades 4 
and 8 in the United States against countries around the world. Since 1995, TIMSS has measured 
worldwide trends in student knowledge of mathematics and science. The next TIMSS assessment will be 
in spring 2015. 


Our school has accepted an invitation from the National Center for Education Statistics (NCES), part of 
the U.S. Department of Education, to participate in TIMSS 2015. {Insert number} of our 8th-grade classes 
will take part. {This/One of these} is your child’s class. The enclosed summary sheet provides some 
background information about TIMSS, explains what is involved for each student selected to participate 
in the study, and gives a contact phone number and email address where you can find answers to any 
questions you might have. 


To have an accurate picture of what U.S. 8th-graders can do in mathematics and science, it is important 
that each student selected take part in the study. In addition to answering mathematics and science 
questions, students will be asked to complete a brief questionnaire about themselves. | urge you to 
support this effort by encouraging your child to take part; however, participation in this study is entirely 
voluntary. Previous experience suggests that students enjoy taking part, and participating students will 
receive a small gift that we think they will like. 


All of the information collected is safeguarded, as required by law. NCES is authorized to conduct this 
study under the Education Sciences Reform Act of 2002 (ESRA, 2002), 20 U.S. Code, Section 9543. Under 
that law, the data provided by schools, staff, and students may be used only for statistical purposes and 
may not be disclosed or used in identifiable form for any other purpose except as required by law (20 
U.S.C., § 9573). Students and schools are never identified in any reports. All reported statistics refer to 
the United States as a whole. 


Thank you for taking the time to learn about this important study. 
Sincerely, 
[School Official] 


Enclosure: 
Facts for Parents: TIMSS 2015 
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A-8. TIMSS 2015 Parent Notification Letter: Grade 8 (Spanish version) 


SCHOOL LETTERHEAD 


Estimado padre/madre o tutor, 


Por medio de la presente queremos informarle acerca de un importante estudio internacional que se 
realizara en nuestra escuela esta primavera acerca del aprendizaje de los estudiantes. El Estudio 
Internacional sobre las Tendencias en Matematicas y Ciencias (TIMSS, por sus siglas en inglés) 
proporciona informacion importante para evaluar el rendimiento de los estudiantes de 4° y 8° grado en 
Estados Unidos en matematicas y ciencias en comparacidn con otros paises del mundo. Desde 1995, 
TIMSS ha medido las tendencias a nivel mundial en el aprendizaje de los estudiantes de matematicas y 
ciencias. 


Nuestra escuela ha aceptado una invitacion del Centro Nacional de Estadisticas de la Educacion (NCES, 
por sus siglas en inglés), parte del Departamento de Educacidn de Estados Unidos, para participar en 
TIMSS 2015. Participaran {Insert number} de nuestras clases de 8° grado. {Esta/Una de estas} es la clase 
de su hijo. El resumen adjunto ofrece informacidn de trasfondo de TIMSS, explica lo que implica la 
participacion en el estudio para cada estudiante seleccionado e incluye un numero de teléfono y un 
correo electronico de contacto donde usted podra encontrar respuestas a cualquier pregunta que tenga. 


Para tener una imagen precisa de lo que los estudiantes de 8° grado en Estados Unidos pueden hacer en 
matematicas y ciencias, es importante que cada estudiante seleccionado participe en el estudio. Ademas 
de contestar las preguntas de matematicas y ciencias, a los estudiantes se les pedira que contesten un 
cuestionario corto acerca de ellos mismos. Quisiéramos pedirle que apoye esta iniciativa animando a su 
hijo a participar; sin embargo, la participacidn en este estudio es completamente voluntaria. Las 
experiencias anteriores sugieren que los estudiantes disfrutan de la participacién. Ademas, los 
estudiantes que participen recibiran un pequefio regalo que creemos les gustara. 


Toda la informaci6n que se retina esta protegida, como lo exige la ley. NCES esta autorizado a realizar 
este estudio de acuerdo con la Ley de Reforma de Ciencias de la Educaci6n de 2002 (ESRA, 2002), Cédigo 
de Estados Unidos, Titulo 20, Secci6n 9543. Segun esa ley, la informacidn proporcionada por las 
escuelas, el personal y los estudiantes Unicamente se puede usar con fines estadisticos y no se puede 
dar a conocer ni usar de alguna manera que pueda identificar a los participantes para ningun otro fin, 
excepto cuando lo exige la ley (20 U.S.C., § 9573). En los informes nunca se identifica ni a los estudiantes 
nia las escuelas. Todas las estadisticas publicadas se refieren a Estados Unidos en conjunto. 


Gracias por tomarse el tiempo de informarse acerca de este importante estudio. 
Atentamente, 
[School Official] 


Documento anexo: 
Informacion para los padres: TIMSS 2015 
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A-9. TIMSS Advanced 2015 Parent Notification Letter 


SCHOOL LETTERHEAD 


Dear Parent or Guardian, 


This letter is to inform you about an important international study of student learning being conducted 
in our school this spring. The Trends in International Mathematics and Science Study (TIMSS) provides 
important information for benchmarking student performance in mathematics and science in the United 
States against countries around the world. Since 1995, TIMSS has measured worldwide trends in 
student knowledge of mathematics and science at grades 4 and 8. The next TIMSS assessment will be in 
spring 2015 and will include TIMSS Advanced at grade 12. 


TIMSS Advanced measures advanced mathematics and physics achievement for students in their final 
year of secondary school. It will provide information about how many students are excelling at highly 
specialized science and mathematics content in comparison with countries around the world. In the 
United States, TIMSS Advanced students have taken or are taking calculus or an advanced physics 
course. 


Our school has accepted an invitation from the National Center for Education Statistics (NCES), U.S. 
Department of Education, to participate in TIMSS Advanced 2015. A select number of our high-achieving 
students will participate, along with your teenager. The enclosed summary sheet provides some 
background information about TIMSS Advanced, explains what is involved for each student selected to 
participate in the study, and gives a contact phone number and email address where you can find 
answers to any questions you might have. 


To have an accurate picture of what U.S. 12th-graders who have taken calculus or advanced physics can 
do, it is important that each student selected take part in the study. In addition to answering 
mathematics and physics questions, students will be asked to complete a brief questionnaire about 
themselves. | urge you to support this effort by encouraging your teenager to take part; however, 
participation in this study is entirely voluntary. Previous experience suggests that students enjoy taking 
part, and participating students will receive a small gift that we think they will like. 


All of the information collected is safeguarded, as required by law. NCES is authorized to conduct this 
study under the Education Sciences Reform Act of 2002 (ESRA, 2002), 20 U.S. Code, Section 9543. Under 
that law, the data provided by schools, staff, and students may be used only for statistical purposes and 
may not be disclosed or used in identifiable form for any other purpose except as required by law (20 
U.S.C., § 9573). Students and schools are never identified in any reports. All reported statistics refer to 
the United States as a whole. 


Thank you for taking the time to learn about this important study. 
Sincerely, 
[School Official] 


Enclosure: 
Facts for Parents: TIMSS Advanced 2015 
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A-10. TIMSS Advanced 2015 Parent Notification Letter (Spanish version) 


SCHOOL LETTERHEAD 


Estimado padre/madre o tutor, 


Por medio de la presente queremos informarle acerca de un importante estudio internacional que se 
realizara en nuestra escuela esta primavera acerca del aprendizaje de los estudiantes. El Estudio 
Internacional sobre las Tendencias en Matematicas y Ciencias (TIMSS, por sus siglas en inglés) 
proporciona informacion importante para evaluar el rendimiento de los estudiantes en Estados Unidos 
en matematicas y ciencias en comparacion con otros paises del mundo. Desde 1995, TIMSS ha medido 
las tendencias a nivel mundial en el aprendizaje de los estudiantes de matematicas y ciencias en 4° y 8° 
grado. La proxima evaluacién TIMSS en la primavera de 2015 incluira TIMSS “Advanced” para 12° grado. 


TIMSS “Advanced” mide los logros de los estudiantes en matematicas y fisica avanzadas en su ultimo 
afio de escuela secundaria. Proporcionara informacion acerca de cuantos estudiantes se distinguen en 
cursos altamente especializados de ciencias y matematicas en comparacion con otros paises del mundo. 
En Estados Unidos, los estudiantes en TIMSS Advanced han tomado o estan tomando un curso de 
calculo o de fisica avanzada. 


Nuestra escuela ha aceptado una invitacion del Centro Nacional de Estadisticas de la Educacion (NCES, 
por sus siglas en inglés) del Departamento de Educacién de Estados Unidos para participar en TIMSS 
“Advanced” 2015. Participara un numero selecto de nuestros estudiantes de alto rendimiento, junto con 
su hijo. El resumen adjunto ofrece informacidn de trasfondo de TIMSS “Advanced”, explica lo que 
implica la participacién en el estudio para cada estudiante seleccionado e incluye un numero de 
teléfono y un correo electrdnico de contacto donde usted podra encontrar respuestas a cualquier 
pregunta que tenga. 


Para tener una imagen precisa de lo que pueden hacer los estudiantes de 12° grado en Estados Unidos 
que han tomado calculo o fisica avanzada, es importante que cada estudiante seleccionado participe en 
el estudio. Ademas de contestar las preguntas de matematicas y fisica, a los estudiantes se les pedira 
que contesten un cuestionario corto acerca de ellos mismos. Quisiéramos pedirle que apoye esta 
iniciativa animando a su hijo a participar; sin embargo, la participacidn en este estudio es 
completamente voluntaria. Las experiencias anteriores sugieren que los estudiantes disfrutan de la 
participaci6n. Ademas, los estudiantes que participen recibiran un pequefio regalo que creemos les 
gustara. 


Toda la informacion que se retina esta protegida, como lo exige la ley. NCES esta autorizado a realizar 
este estudio de acuerdo con la Ley de Reforma de Ciencias de la Educacién de 2002 (ESRA, 2002), Codigo 
de Estados Unidos, Titulo 20, Seccidn 9543. Segun esa ley, la informacién proporcionada por las 
escuelas, el personal y los estudiantes Unicamente se puede usar con fines estadisticos y no se puede 
dar a conocer ni usar de alguna manera que pueda identificar a los participantes para ningun otro fin, 
excepto cuando lo exige la ley (20 U.S.C., § 9573). En los informes nunca se identifica ni a los estudiantes 
nia las escuelas. Todas las estadisticas publicadas se refieren a Estados Unidos en conjunto. 
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A-10. TIMSS Advanced 2015 Parent Notification Letter (Spanish version)—Continued 


Gracias por tomarse el tiempo de informarse acerca de este importante estudio. 


Atentamente, 


[School Official] 
Documento anexo: 
Informacién para los padres: TIMSS Advanced 2015 
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A-11. TIMSS 2015 Implicit Consent Letter: Grade 4 


SCHOOL LETTERHEAD 


Dear Parent or Guardian, 


This letter is to inform you about an important international study of student learning being conducted 
in our school this spring. The Trends in International Mathematics and Science Study (TIMSS) provides 
important information for benchmarking student performance in mathematics and science at grades 4 
and 8 in the United States against countries around the world. Since 1995, TIMSS has measured 
worldwide trends in student knowledge of mathematics and science. The next TIMSS assessment will be 
in the spring of 2015. 


Our school has accepted an invitation from the National Center for Education Statistics (NCES), part of 
the U.S. Department of Education, to participate in the TIMSS 2015 main study. {Insert number} of our 
4th grade classes will take part. {This/One of these} is your child’s class. The enclosed summary sheet 
provides some background information about TIMSS, explains what is involved for each student selected 
to participate in the study, and gives a contact phone number and email address where you can find 
answers to any questions you might have. 


To have an accurate picture of what U.S. 4th graders can do in mathematics and science, it is important 
that each student selected take part in the study. In addition to answering mathematics and science 
questions, students will be asked to complete a brief questionnaire about themselves. | urge you to 
support this effort by encouraging your child to take part; however, participation in this study is entirely 
voluntary. Previous experience suggests that students actually enjoy taking part, and participating 
students will receive a small gift, which we think they will like. 


All of the information collected is safeguarded, as required by law. NCES is authorized to conduct this 
study under the Education Sciences Reform Act of 2002 (ESRA, 2002), 20 U.S. Code, Section 9543. Under 
that law, the data provided by schools, staff, and students may be used only for statistical purposes and 
may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 
U.S.C., § 9573). Students and schools are never identified in any reports. All reported statistics refer to 
the United States as a whole. 


If you have any objection to your child joining in the TIMSS activities, please let us know by completing 
the attached consent form and returning it to the school. 


Thank you for taking the time to learn about this important study. 


Sincerely, 


[School Official] 

Enclosures: 

Facts for Parents about the TIMSS 2015 Main Study 
Parent/Guardian Consent Form 
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A-12. TIMSS 2015 Implicit Consent Letter: Grade 8 


SCHOOL LETTERHEAD 


Dear Parent or Guardian, 


This letter is to inform you about an important international study of student learning being conducted 
in our school this spring. The Trends in International Mathematics and Science Study (TIMSS) provides 
important information for benchmarking student performance in mathematics and science at grades 4 
and 8 in the United States against countries around the world. Since 1995, TIMSS has measured 
worldwide trends in student knowledge of mathematics and science. The next TIMSS assessment will be 
in the spring of 2015. 


Our school has accepted an invitation from the National Center for Education Statistics (NCES), part of 
the U.S. Department of Education, to participate in the TIMSS 2015 main study. {Insert number} of our 
8th grade classes will take part. {This/One of these} is your child’s class. The enclosed summary sheet 
provides some background information about TIMSS, explains what is involved for each student selected 
to participate in the study, and gives a contact phone number and email address where you can find 
answers to any questions you might have. 


To have an accurate picture of what U.S. 8th graders can do in mathematics and science, it is important 
that each student selected take part in the study. In addition to answering mathematics and science 
questions, students will be asked to complete a brief questionnaire about themselves. | urge you to 
support this effort by encouraging your child to take part; however, participation in this study is entirely 
voluntary. Previous experience suggests that students actually enjoy taking part, and participating 
students will receive a small gift, which we think they will like. 


All of the information collected is safeguarded, as required by law. NCES is authorized to conduct this 
study under the Education Sciences Reform Act of 2002 (ESRA, 2002), 20 U.S. Code, Section 9543. Under 
that law, the data provided by schools, staff, and students may be used only for statistical purposes and 
may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 
U.S.C., § 9573). Students and schools are never identified in any reports. All reported statistics refer to 
the United States as a whole. 


If you have any objection to your child joining in the TIMSS activities, please let us know by completing 
the attached consent form and returning it to the school. 


Thank you for taking the time to learn about this important study. 
Sincerely, 

[School Official] 

Enclosures: 


Facts for Parents about the TIMSS 2015 Main Study 
Parent/Guardian Consent Form 
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A-13. TIMSS Advanced 2015 mplicit Consent Letter 


SCHOOL LETTERHEAD 


Dear Parent or Guardian, 


This letter is to inform you about an important international study of student learning being conducted 
in our school this spring. The Trends in International Mathematics and Science Study (TIMSS) provides 
important information for benchmarking student performance in mathematics and science in the United 
States against countries around the world. Since 1995, TIMSS has measured worldwide trends in 
student knowledge of mathematics and science at grades 4 and 8. The next TIMSS assessment will be in 
the spring of 2015 and will include TIMSS Advanced at grade 12. 


TIMSS Advanced measures advanced mathematics and physics achievement for students in their final 
year of secondary school. It will provide information about how many students are excelling at highly 
specialized science and mathematics content in comparison with countries around the world. In the 
United States, TIMSS Advanced students have taken or are taking calculus, or an advanced physics course. 


Our school has accepted an invitation from the National Center for Education Statistics (NCES), U.S. 
Department of Education, to participate in the TIMSS Advanced main study. A select few of our high- 
achieving students will participate, along with your teenager. The enclosed summary sheet provides 
some background information about TIMSS Advanced, explains what is involved for each student 
selected to participate in the study, and gives a contact phone number and email address where you can 
find answers to any questions you might have. 


To have an accurate picture of what U.S. 12" graders who have taken calculus or advanced physics can 
do, it is important that each student selected take part in the study. In addition to answering 
mathematics and physics questions, students will be asked to complete a brief questionnaire about 
themselves. | urge you to support this effort by encouraging your teenager to take part; however, 
participation in this study is entirely voluntary. Previous experience suggests that students actually enjoy 
taking part, and participating students will receive a small gift, which we think they will like. 


All of the information collected is safeguarded, as required by law. NCES is authorized to conduct this 
study under the Education Sciences Reform Act of 2002 (ESRA, 2002), 20 U.S. Code, Section 9543. Under 
that law, the data provided by schools, staff, and students may be used only for statistical purposes and 
may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 
U.S.C., § 9573). Students and schools are never identified in any reports. All reported statistics refer to 
the United States as a whole. 


If you have any objection to your teenager joining in the TIMSS activities, please let us know by 
completing the attached consent form and returning it to the school. 


Thank you for taking the time to learn about this important study. 


Sincerely, 


[School Official] 

Enclosures: 

Facts for Parents about the TIMSS Advanced Main Study 
Parent/Guardian Consent Form 
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A-14. TIMSS 2015 Implicit Consent Form: Grades 4 and 8 


Trends in International Mathematics and Science Study (TIMSS) 
Main Study 
Parent/Guardian Consent Form 


Your child has been asked to participate in an international study of student learning called the 
Trends in International Mathematics and Science Study (TIMSS). Each student who participates will 
receive a small gift. This assessment will be conducted by a team of researchers from Westat, who 
are operating under contract on behalf of the U. S. Department of Education. In spring 2015, new 
4*_ and 8"-grade questions will be tested in schools in the United States. 


If you grant permission for your child to participate in the 2015 TIMSS, you do not need to 
return this form. 


If you do not consent to your child’s participation in the 2015 TIMSS, please return this form 
to your child’s school as soon as possible. 


I do not grant permission for my child, , to participate in 


the Trends in International Mathematics and Science Study Main Study. 


(Signature of parent or guardian) 


Date of signature: / / 


PLEASE PRINT: 


Student name: 


School name: 


FOR OFFICE USE ONLY: 


Student ID: 
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A-15. TIMSS Advanced 2015 Implicit Consent Form 


Trends in International Mathematics and Science Study (TIMSS) Advanced 
Main Study 
Parent/Guardian Consent Form 


Your teenager has been asked to participate in the Main Study of an international study of student 
learning called the Trends in International Mathematics and Science Study (TIMSS) Advanced. Each 
student who participates will receive a small gift. This assessment will be conducted by a team of 
researchers from Westat, who are operating under contract on behalf of the U. S. Department of 
Education. In spring 2015, TIMSS Advanced will measure advanced mathematics and physics 
achievement for students in their final year of secondary school. It will provide information about 
how many students are excelling at highly specialized science and mathematics content in 
comparison with countries around the world. 


If you grant permission for your child to participate in the TIMSS Advanced main study, 
you do not need to return this form. 


If you do not consent to your child’s participation in the TIMSS Advanced main study, 
please return this form to your child’s school as soon as possible. 


I do not grant permission for my child, , to participate in 
the Trends in International Mathematics and Science Study Main Study. 


(Signature of parent or guardian) 


Date of signature: / / 


PLEASE PRINT: 


Student name: 


School name: 


FOR OFFICE USE ONLY: 


Student ID: 
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A-16. TIMSS 2015 Explicit Consent Letter: Grade 4 


SCHOOL LETTERHEAD 


Dear Parent or Guardian, 


This letter is to inform you about an important international study of student learning being conducted 
in our school this spring. The Trends in International Mathematics and Science Study (TIMSS) provides 
important information for benchmarking student performance in mathematics and science at grades 4 
and 8 in the United States against countries around the world. Since 1995, TIMSS has measured 
worldwide trends in student knowledge of mathematics and science. The next TIMSS assessment will be 
in the spring of 2015. 


Our school has accepted an invitation from the National Center for Education Statistics (NCES), part of 
the U.S. Department of Education, to participate in the TIMSS main study. {Insert number} of our 4¢- 
grade classes will take part. {This/One of these} is your child’s class. The enclosed summary sheet 
provides some background information about TIMSS, explains what is involved for each student selected 
to participate in the study, and gives a contact phone number and email address where you can find 
answers to any questions you might have. 


To have an accurate picture of what U.S. 4th graders can do in mathematics and science, it is important 
that each student selected take part in the study. In addition to answering mathematics and science 
questions, students will be asked to complete a brief questionnaire about themselves. | urge you to 
support this effort by encouraging your child to take part; however, participation in this study is entirely 
voluntary. Previous experience suggests that students actually enjoy taking part, and participating 
students will receive a small gift, which we think they will like. 


All of the information collected is safeguarded, as required by law. NCES is authorized to conduct this 
study under the Education Sciences Reform Act of 2002 (ESRA, 2002), 20 U.S. Code, Section 9543. Under 
that law, the data provided by schools, staff, and students may be used only for statistical purposes and 
may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 
U.S.C., § 9573). Students and schools are never identified in any reports. All reported statistics refer to 
the United States as a whole. 


Before we can allow your child to join in the TIMSS activities, we must have your written consent. 
Please let us know by completing the attached form and returning it to the school. 


Thank you for taking the time to learn about this important study and to consider your child’s 
participation in it. 


Sincerely, 


[School Official] 

Enclosures: 

Facts for Parents about the TIMSS 2015 Main Study 
Parent/Guardian Consent Form 
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A-17. TIMSS 2015 Explicit Consent Letter: Grade 8 


SCHOOL LETTERHEAD 


Dear Parent or Guardian, 


This letter is to inform you about an important international study of student learning being conducted 
in our school this spring. The Trends in International Mathematics and Science Study (TIMSS) provides 
important information for benchmarking student performance in mathematics and science at grades 4 
and 8 in the United States against countries around the world. Since 1995, TIMSS has measured 
worldwide trends in student knowledge of mathematics and science. The next TIMSS assessment will be 
in the spring of 2015. 


Our school has accepted an invitation from the National Center for Education Statistics (NCES), part of 
the U.S. Department of Education, to participate in the TIMSS 2015 main study. {Insert number} of our 
8th-grade classes will take part. {This/One of these} is your child’s class. The enclosed summary sheet 
provides some background information about TIMSS, explains what is involved for each student selected 
to participate in the study, and gives a contact phone number and email address where you can find 
answers to any questions you might have. 


To have an accurate picture of what U.S. 8th graders can do in mathematics and science, it is important 
that each student selected take part in the study. In addition to answering mathematics and science 
questions, students will be asked to complete a brief questionnaire about themselves. | urge you to 
support this effort by encouraging your child to take part; however, participation in this study is entirely 
voluntary. Previous experience suggests that students actually enjoy taking part, and participating 
students will receive a small gift, which we think they will like. 


All of the information collected is safeguarded, as required by law. NCES is authorized to conduct this 
study under the Education Sciences Reform Act of 2002 (ESRA, 2002), 20 U.S. Code, Section 9543. Under 
that law, the data provided by schools, staff, and students may be used only for statistical purposes and 
may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 
U.S.C., § 9573). Students and schools are never identified in any reports. All reported statistics refer to 
the United States as a whole. 


Before we can allow your child to join in the TIMSS activities, we must have your written consent. 
Please let us know by completing the attached form and returning it to the school. 


Thank you for taking the time to learn about this important study and to consider your child’s 
participation in it. 


Sincerely, 


[School Official] 

Enclosures: 

Facts for Parents about the TIMSS 2015 Main Study 
Parent/Guardian Consent Form 
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A-18. TIMSS Advanced 2015 Explicit Consent Letter 


SCHOOL LETTERHEAD 


Dear Parent or Guardian, 


This letter is to inform you about an important international study of student learning being conducted in our 
school this spring. The Trends in International Mathematics and Science Study (TIMSS) provides important 
information for benchmarking student performance in mathematics and science in the United States against 
countries around the world. Since 1995, TIMSS has measured worldwide trends in student knowledge of 
mathematics and science at grades 4 and 8. The next TIMSS assessment will be in the spring of 2015, and 
will include a TIMSS Advanced component at grade 12. 


TIMSS Advanced measures advanced mathematics and physics achievement for students in their final year of 
secondary school. It will provide information about how many students are excelling at highly specialized 
science and mathematics content in comparison with countries around the world. In the United States, 


TIMSS Advanced students have taken or are taking calculus, or an advanced physics coutse. 


Our school has accepted an invitation from the National Center for Education Statistics (NCES), U.S. 
Department of Education, to participate in the TIMSS Advanced main study. A select few of our high 
achieving students will participate, along with your teenager. The enclosed summary sheet provides some 
background information about TIMSS Advanced, explains what is involved for each student selected to 
participate in the study, and gives a contact phone number and email address where you can find answers to 


any questions you might have. 


To have an accurate picture of what U.S. 12% graders who have taken calculus or advanced physics can do, it 
is important that each student selected take part in the study. In addition to answering mathematics and 
physics questions, students will be asked to complete a brief questionnaire about themselves. I urge you to 
support this effort by encouraging your teenager to take part; however, participation in this study is entirely 
voluntary. Previous experience suggests that students actually enjoy taking part, and participating students will 
receive a small gift, which we think they will like. 


All of the information collected is safeguarded, as required by law. NCES is authorized to conduct this study 
under the Education Sciences Reform Act of 2002 (ESRA, 2002), 20 U.S. Code, Section 9543. Under that law, 
the data provided by schools, staff, and students may be used only for statistical purposes and may not be 
disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C., § 9573). 
Students and schools are never identified in any reports. All reported statistics refer to the United States as a 


whole. 


Before we can allow your teenager to join in the TIMSS activities, we must have your written consent. Please 
let us know by completing the attached form and returning it to the school. 


Thank you for taking the time to learn about this important study and to consider your teenager’s participation 
in it, 

Sincerely, 

[School Official] 

Enclosures: 


Facts for Parents about the TIMSS Advanced Main Study 
Parent/Guardian Consent Form 
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A-19. TIMSS 2015 Explicit Consent Form: Grades 4 and 8 


Trends in International Mathematics and Science Study (TIMSS) 
Main Study 
Parent/Guardian Consent Form 


Your child has been asked to participate in an international study of student learning called the 2015 
Trends in International Mathematics and Science Study (TIMSS). Each student who participates will 
receive a small gift. This assessment will be conducted by a team of researchers from Westat, who 
are operating under contract on behalf of the U. S. Department of Education. 


a Yes, I grant permission for my child to participate in the 2015 TIMSS. 


L No, I do not grant permission for my child to participate in the 2015 TIMSS. 


(Signature of parent or guardian) 


Date of signature: / f 


PLEASE PRINT: 


Student name: 


School name: 


FOR OFFICE USE ONLY: 


Student ID: 
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A-20. TIMSS Advanced 2015 Explicit Consent Form 


Trends in International Mathematics and Science Study (TIMSS) Advanced 
Main Study 
Parent/Guardian Consent Form 


Your teenager has been asked to participate in the Main Study of an international study of student 
learning called the Trends in International Mathematics and Science Study (TIMSS) Advanced. Each 
student who participates will receive a small gift. This assessment will be conducted by a team of 
researchers from Westat, who are operating under contract on behalf of the U. S. Department of 
Education. In spring 2015, TIMSS Advanced will measure advanced mathematics and physics 
achievement for students in their final year of secondary school. It will provide information about 
how many students are excelling at highly specialized science and mathematics content in 
comparison with countries around the world. In the United States, TIMSS Advanced students have 
taken or are taking calculus, or an advanced physics course. 


[| Yes, I grant permission for my teenager to participate in the TIMSS Advanced Main Study. 


ir No, I do not grant permission for my teenager to participate in the TIMSS Advanced Main 
Study. 


(Signature of parent or guardian) 


Date of signature: FA / 


PLEASE PRINT: 


Student name: 


School name: 


FOR OFFICE USE ONLY: 


Student ID: 
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A-21. Facts About TIMSS 2015 for Parents 


Between March and May 2015, your child’s school will be one of several hundred schools nationwide 
taking part in the Trends in International Mathematics and Science Study (TIMSS). The schools were 
selected randomly to represent the nation’s schools and, within each school, fourth- or eighth-grade 
students were selected randomly to represent the nation’s fourth- or eighth-graders. Your child was 
among those students selected to take part in TIMSS. 


What is TIMSS? 


TIMSS is an international assessment that measures student learning in mathematics and science. Every 
4 years since 1995, TIMSS documents worldwide trends in the mathematics and science achievement of 
fourth and eighth-graders. The National Center for Education Statistics within the U.S. Department of 
Education sponsors U.S. participation in TIMSS. Along with more than 50 other countries, the United 
States will take part in the 2015 TIMSS cycle as we did in 1995, 1999, 2003, 2007, and 2011. 


What is the purpose of TIMSS 2015? 


TIMSS measures trends in mathematics and science achievement at the fourth- and eighth-grade levels 
as well as collects information about school and teacher practices related to instruction. It provides a 
unique opportunity to compare U.S. students’ math and science knowledge and skills at the fourth- and 
eighth-grade levels with that of their peers in countries around the world. TIMSS complements what we 
learn from national assessments by identifying the strengths and weaknesses of student performance 
relative to students around the world. The results inform national discussions about education as well as 
international competitiveness. 


What is involved? 


TIMSS staff will visit the school and administer the study, which will take approximately 2% hours. The 
assessment itself is 90 minutes, with breaks between sections. Students will also be asked some 
questions about themselves and their educational experience. 


What are the benefits? 


The nation as a whole benefits from TIMSS by having a greater understanding of how the knowledge 
and skills of U.S. fourth- and eighth-graders compare with fourth- and eighth-graders from other 
countries. Schools that participate in TIMSS will receive $200, and each student who participates will 
receive a small gift that we believe they will like. 
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A-21. Facts About TIMSS 2015 for Parents—Continued 


Who administers TIMSS 2015? 


The entire assessment is administered by trained staff from Westat, a research organization under 
contract to the U.S. Department of Education’s National Center for Education Statistics. All of the 
information collected is safeguarded, as required by law. NCES is authorized to conduct this study under 
the Education Sciences Reform Act of 2002 (ESRA, 2002), 20 U.S. Code, Section 9543. Under that law, 
the data provided by schools, staff, and students may be used only for statistical purposes and may not 
be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C., § 
9573). Students and schools are never identified in any reports. All reported statistics refer to the United 
States as a whole. 


Where can | find out more about TIMSS? 


More information about TIMSS is available at the TIMSS website at http://nces.ed.gov/timss or 
http://timss.bc.edu. If you would like to contact a TIMSS staff member directly, please feel free to call 
the TIMSS hotline at 855-445-5604 or email us at TIMSS@westat.com. 


O.M.B. No. 1850-0695 
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A-22. Facts About TIMSS Advanced 2015 for Parents 


Between March and May 2015, your teenager’s school will be one of several hundred schools 
nationwide taking part in a special component of the Trends in International Mathematics and Science 
Study (TIMSS), called TIMSS Advanced. The schools were selected randomly to represent the nation’s 
schools that offer courses in advanced mathematics and physics. Within each school, twelfth-grade 
students were selected randomly to represent the nation’s twelfth-graders who have taken or are taking 
advanced mathematics, such as calculus, or advanced physics. Your teenager was among those students 
selected to take part in TIMSS Advanced. 


What is TIMSS? 


TIMSS is an international assessment that measures student learning in mathematics and science. Every 
4 years since 1995, TIMSS documents worldwide trends in the mathematics and science achievement of 
fourth- and eighth-graders. The National Center for Education Statistics within the U.S. Department of 
Education sponsors U.S. participation in TIMSS. Along with more than 50 other countries, the United 
States will take part in the 2015 TIMSS cycle as we did in 1995, 1999, 2003, 2007, and 2011. TIMSS 2015 
will include the TIMSS Advanced component, which will be administered to twelfth-graders. 


What is the purpose of the TIMSS Advanced? 


TIMSS Advanced measures advanced mathematics and physics achievement for students in their final 
year of secondary school. It will provide information about how many students are excelling at highly 
specialized science and mathematics content in comparison with countries around the world. In the 
United States, TIMSS Advanced students have taken or are taking calculus or an advanced physics 
course. 


What is involved? 


TIMSS staff will visit the school and administer the study, which will take approximately 2% hours. The 
assessment itself is 90 minutes, with breaks between sections. Students will also be asked some 
questions about themselves and their educational experience. 


What are the benefits? 


The nation as a whole benefits from TIMSS by having a greater understanding of how the knowledge 
and skills of the best prepared U.S. twelfth-graders compare with twelfth-graders from other countries 
who have taken specialized mathematics and science courses. Schools that participate in TIMSS will 
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A-22. Facts About TIMSS Advanced 2015 for Parents—Continued 


receive $200, and students who participate will receive a small gift that we believe they will like. 
Participating students will also receive a volunteer service certificate of 4 hours from the U.S. 
Department of Education. 


Who administers TIMSS Advanced? 


The entire assessment is administered by trained staff from Westat, a research organization under 
contract to the U.S. Department of Education’s National Center for Education Statistics. All of the 
information collected is safeguarded, as required by law. NCES is authorized to conduct this study under 
the Education Sciences Reform Act of 2002 (ESRA, 2002), 20 U.S. Code, Section 9543. Under that law, 
the data provided by schools, staff, and students may be used only for statistical purposes and may not 
be disclosed or used in identifiable form for any other purpose except as required by law (20 U.S.C., § 
9573). Students and schools are never identified in any reports. All reported statistics refer to the United 
States as a whole. 


Where can | find out more about TIMSS and TIMSS Advanced? 


More information about TIMSS is available at the TIMSS website at http://nces.ed.gov/timss or 
http://timss.bc.edu. If you would like to contact a TIMSS staff member directly, please feel free to call 
the TIMSS hotline at 855-445-5604 or email us at TIMSS@westat.com. 


O.M.B. No. 1850-0695 
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Exhibit B-1. TIMSS 2015 Brochure 


Other information collected by TIMSS 


TIMSS is more than an assessment of student 
knowledge in mathematics and science. TIMSS 
also considers the context in which learning 
occurs. Students, teachers, and schools are asked 
about a variety of aspects of the environments in 
which content is taught, learned, practiced, and 
applied. In this way, TIMSS provides each country 
with a rich source of information on the factors 
influencing mathematics and science achievement. 


Participating countries and other 
education systems in TIMSS 2015 


North and Netherlands Kazakhstan 
South America Northern Ireland Korea, Rep. of 
Canada Norway Kuwait 
Chile Poland Lebanon 
United States Portugal Malaysia 
Europe Russian Federation Oman 
iAustria Slovak Republic Palestinian 
Belgium (Flemish) Slovenia Nat'l Auth. 
Bulgaria Spain Qatar 
Croatia Sweden Saudi Arabia 
Cyprus Turkey Singapore 
Czech Republic Ukraine Thailand 
Denmark Asia and United Arab 
England Middle East Emirates 
Finland Armenia Africa 

France Bahrain Botswana 
Georgia Chinese Taipei Egypt 
Germany Hong Kong SAR South Africa 
Hungary Iran, Islamic : 
Ireland Rep. of Oeeong 
Italy Israel Australia 
Lithuania Tepes New Zealand 
Malta Jordan 


Benchmarking participants 


Abu Dhabi, UAE Dubai, UAE 
Alberta, Canada Ontario, Canada 
Buenos Aires, Argentina Quebec, Canada 


NCES is authorized to conduct TIMSS 
under the Education Sciences Reform Act 
of 2002 (ESRA 2002), 20 U.S. Code, § 9543. 
Information collected will help the U.S. 
Department of Education’s ongoing efforts 
to benchmark student achievement in 

the United States. Participation is voluntary. 
By law, data collected may be used only 
for statistical purposes and may not be 
disclosed, or used, in identifiable form 

for any other purpose except as required 
by law (20 U.S. Code, § 9573). The U.S. 
Office of Management and Budget has 
approved the data collection under 

OMB # 1850-0695. Individual responses 

will be combined with those from other 
participants to produce summary statistics 
and reports. 


@ 
e 
e | e S NATIONAL CENTER For 
e EDUCATION STATISTICS 


Institute of Education Sciences 


For questions about TIMSS 2015, 
contact the toll-free TIMSS information 
hotline at 855-445-5604 or email 
TIMSS@westat.com. 


Trends in International 
Mathematics and 
Science Study (TIMSS) 


| eC S NATIONAL CENTER For 
EDUCATION STATISTICS 


Institute of Educ ° ciences 
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Exhibit B-1. 


What is TIMSS? 


The Trends in International Mathematics and 
Science Study (TIMSS) is an international 
assessment and research project designed to 
measure trends in mathematics and science 
achievement at the fourth- and eighth-grade 
levels as well as collect information about school 
and teacher practices related to instruction. Since 
1995, TIMSS has been administered every 4 years. 
TIMSS 2015, the sixth study in the series, will 
involve students from more than 50 countries, 
including the United States. 


TIMSS is sponsored by the International 
Association for the Evaluation of Educational 
Achievement (IEA) and managed in the United 
States by the National Center for Education 
Statistics (NCES), part of the U.S. Department 
of Education. 


Why is TIMSS important? 


TIMSS provides a unique opportunity to compare 
USS. students’ math and science knowledge 

and skills at the fourth- and eighth-grade levels 
with that of their peers in countries around the 
world. TIMSS complements what we learn from 
national assessments by identifying the strengths 
and weaknesses of student performance relative 

to students around the world. The results inform 
national discussions about education as well as 
international competitiveness. 


TIMSS provides valuable benchmark information 
on how U.S. students compare to students around 
the world, allows educators and policymakers to 
examine other educational systems for practices 
that could have application to the United States, 
and contributes to ongoing discussions of ways 

to improve the quality of education of all students. 


TIMSS 2015 Brochure—Continued 


What type of assessment 
is TIMSS? 


The TIMSS mathematics and science assessment 
is developed through an international consensus- 
building process involving input from U.S. and 
international experts in mathematics, science, 
and measurement. In a final step, the assessment 


Key findings from TIMSS 2011 


Mathematics 


e At grade 4, the U.S. average mathematics 
score (541) was higher than the TIMSS scale 
average of 500. The United States was 
among the top 15 education systems 
(8 education systems had higher averages 
and 6 were not measurably different) 
and scored higher, on average, than 42 
education systems. The U.S. mathematics 
average at grade 4 rose 12 points between 
2007 and 2011.The United States was one 
of 12 education systems that increased 
its average score during this time period. 


At grade 8, the U.S. mathematics score 
(509) was higher than the TIMSS scale 
average of 500. The United States was 
among the top 24 education systems 

in mathematics (11 education systems 
had higher averages and 12 were not 
measurably different) and scored higher, 
on average, than 32 education systems. 
The U.S. mathematics average score 

at grade 8 was not measurably different 
than in 2007. 


is endorsed as suitable by all participating countries. 
The assessment contains a mix of questions; some 
require students to select appropriate responses 
while others require that students solve problems 
and provide written answers. Examples of released 
TIMSS items are available at http://nces.ed.gov/ 
timss/educators.asp. 


Science 


e At grade 4, the U.S. average science 
score (544) was higher than the TIMSS 
scale average of 500. The United States 
was among the top 10 education 
systems (6 education systems had 
higher averages and 3 were not 
measurably different) and scored 
higher, on average, than 47 education 
systems. The U.S. science average score 
at grade 4 was not measurably different 
than in 2007. 


At grade 8, the U.S. average science 
score (525) was higher than the TIMSS 
scale average of 500. The United States 
was among the top 23 education 
systems (12 education systems had 
higher averages and 10 were not 
measurably different) and scored 
higher, on average, than 33 education 
systems. The U.S. science average score 
at grade 8 was not measurably different 
than in 2007. 


Exhibit B-2. TIMSS 2015 FAQ 


TIMSS 2015 


What is TIMSS? 


The Trends in International Mathematics and Science 
Study (TIMSS) is an international assessment and 
research project designed to measure trends in 
mathematics and science achievement at the fourth- 
and eighth-grade levels as well as collect information 


about school and teacher practices related to instruction. 
Since 1995, TIMSS has been administered every 4 years. 


TIMSS 2015, the sixth study in the series, will involve 
students from more than 50 countries, including the 
United States. 


Why was my school selected 

for participation? 

Schools of varying demographics and locations were 
randomly selected so that the overall U.S. sample is 
representative of the overall U.S. school population. 
‘The random selection process is important for ensuring 
that a country’s sample accurately reflects its schools 
and, therefore, can fairly be compared with samples 
of schools from other countries. 


Will all our fourth- or eighth-grade 
students be asked to participate? 


It depends on the number of fourth- or eighth-grade 
classrooms in the school. In schools with only one 
or two such classrooms, all students will be asked 

to participate. In schools with more than two such 
classrooms, only students in two randomly selected 
classrooms will be asked to participate. In addition, 
some students with special needs or limited English 
proficiency may be excused from the assessment. 


Who conducts the TIMSS assessment? 


‘The entire assessment process will be conducted by 
trained staff from Westat, a research organization 

under contract with the National Center for Education 
Statistics in the U.S. Department of Education. NCES 
is authorized to conduct this study under the Education 


Sciences Reform Act of 2002 (ESRA 2002), 20 U.S. 
Code, § 9543, and approval of the U.S. Office of 
Management and Budget under OMB # 1850-0695. 


How are the teacher and school 
questionnaires administered? 


The teacher and school questionnaires are administered 
online from a secure website. Teacher questionnaires 
take about 35 minutes to complete and ask teachers 
questions about their experience, available resources, 
and instructional practices. School questionnaires take 
about 30 minutes to complete and ask about school 
practices and resources. 


Do teachers need to help administer 
the assessment? 


No. Westat field staff will visit the school on the day 

of the assessment, bringing with them all the materials 
required, and they will handle the entire administration 
of the assessment. 


When will the assessment be conducted? 


‘The assessment will be conducted between March 30 
and May 29, 2015. Each school will be notified of its 
scheduled assessment date in summer 2014. A TIMSS 
representative will work with schools to identify an 
alternate date should there be a conflict on that date. 


Where will the assessment be conducted? 


The assessment will be conducted in the schools that 
are selected to participate. 


How long does the assessment take? 


‘The assessment session is approximately 212 hours 

and includes the administration of the assessment, 

a questionnaire that students complete, and a break 
between the two. The questionnaire takes approximately 
30 minutes to complete. 
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Exhibit B-2. TIMSS 2015 FAQ—Continued 


What will happen with the collected data? 


The data from the assessment will be used to evaluate 
how the knowledge and skills of U.S. students 
compare to those of their peers in other participating 
countries. By law, the data provided by schools, staff, 
and students may be used only for statistical purposes 
and may not be disclosed or used in identifiable 

form for any other purpose except as required by 

law [Education Science Reform Act of 2002 (ESRA 
2002), 20 U.S. Code, Section 9573]. Reports of 

the findings from the assessment will not identify 
participating districts, schools, students, or individual 
staff. Individual responses will be combined with 
those of other participants to produce summary 
statistics and reports. 


OMB # 1850-0695 


Are schools required by federal law 
to participate? 


No. School participation is voluntary. However, 
we hope you will participate in this study so that 
students like those in your school are accurately 
and fairly represented. 


Where can | find more information 
about TIMSS? 


Visit the TIMSS website at http://nces.ed.gov/timss or 
the TIMSS international website at http://timss.bc.edu. 


For additional information about TIMSS 2015, contact 
the toll-free TIMSS information hotline at 855-445-5604 
or email TIMSS@westat.com. 
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Exhibit B-3. TIMSS 2015 Summary of School Activities 


Summary of 
School Activities 
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For questions about TIMSS 2015, 
contact the toll-free TIMSS information 
hotline at 855-445-5604 

or email TIMSS@westat.com 


OMB # 1850-0695 


les 


NATIONAL CENTER for 
EDUCATION STATISTICS 


Summary of 
School Activities 


Trends in International 
Mathematics and 
Science Study (TIMSS) 


2015 
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SUMMARY OF SCHOOL ACTIVITIES: TIMSS 2015 


Principal = Designate a school coordinator 


= Confirm an assessment date convenient 
for your school 


School coordinator 


Teachers of 
sampled classes 


Selected students 


= Complete a brief school questionnaire on the 
characteristics of the school, its enrollment, resources, 
policies, and learning environment (available online 
or hardcopy) 


= Share the importance of participation in the study 
with school staff 


= Arrange assessment day location 
= Confirm date and location with TIMSS assessment staff 


m Provide class lists and student lists through online 
MyTIMSS portal 


= Notify teachers, selected students, and students’ parents 
of the study and benefit of participating 

= Work with assessment staff fo identify students 
with special needs 


= Coordinate the principal's completion of the school 
questionnaire (online or hardcopy) 


= Coordinate the teachers’ completion of the teacher 
questionnaire (online or hardcopy) 


= Distribute parent notification letters to parents 
of selected students 


= Complete online teacher questionnaire 


= Support assessment day activities 


= Encourage students fo participate and do their best 
on the assessment 


= Confirm space for assessment is problem-free 

= Collect complete school and teacher 
questionnaires (if not completed online) 
and give to assessment staff 

= Ensure all sampled students attend the assessment 
session 

= Meet with assessment staff and provide feedback 
about the assessment process 


= Attend the assessment session, complete 
the assessment and student questionnaire 


= Represent other similar U.S. schools 

= Receive a $200 check for the school 

= Receive a confidential report on how 
students selected from your school 
performed on TIMSS assessment questions 
that are made public 


Receive a $100 personal check 
Opportunity to participate in a professional 
development web seminar series 

on international assessments and 
educational policy 


Represent U.S. teachers in the international 
study results 

Opportunity to participate in a professional 
development web seminar series 

on international assessments and 
educational policy 


Receive a small thank-you gift 


Represent the United States in the 
international study 


TIMSS representative = Work with the school to confirm 


an assessment date 


= Help school coordinator with assessment 


details and logistics 


= Protect school and student confidentiality 


= Call the school coordinator to discuss assessment 
day location and student participation 


= Review selected classes with school coordinator 


= Conduct assessment from start to finish 


= Furnish all the assessment materials, pencils, and 
test booklets 


= Conduct a brief debriefing interview with the school 
coordinator at the end of the assessment 


= Maintain security of all materials 


Ensure quality and uniformity of data 
collected across the United States 
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Exhibit B-4. 


Other information collected 
by TIMSS Advanced 


TIMSS Advanced is more than an assessment 

of student knowledge in advanced mathematics 
and physics. TIMSS Advanced also considers 

the context in which learning occurs. Students, 
teachers, and schools are asked about a variety 

of aspects of the environments in which content 
is taught, learned, practiced, and applied. In this 
way, TIMSS Advanced provides each country 
with a rich source of information on the factors 
influencing mathematics and science achievement. 


Participating countries in TIMSS 
Advanced 2015 


France Norway Slovenia 
Georgia Portugal Spain 

Italy Russian Sweden 
Lebanon Federation United States 


TIMSS Advanced 2015 Brochure 


NCES is authorized to conduct TIMSS 
under the Education Sciences Reform Act 
of 2002 (ESRA 2002), 20 U.S. Code, § 9543. 
nformation collected will help the U.S. 
Department of Education's ongoing efforts 
io benchmark student achievement in 

he United States. Participation is voluntary. 
By law, data collected may be used only 
or statistical purposes and may not be 
disclosed or used in identifiable form 

or any other purpose except as required 
by law (20 U.S. Code, § 9573).The U.S. 
Office of Management and Budget has 
approved the data collection under 

OMB #1850-0695. Individual responses 

will be combined with those from other 
participants to produce summary statistics 
and reports. 


e 
e 
e | es NATIONAL CENTER For 
e EDUCATION STATISTICS 


Institute of Education Sciences 


For questions about TIMSS 2015, 
contact the toll-free TIMSS information 
hotline at 855-445-5604 or email 
TIMSS@westat.com. 


Trends in International 
Mathematics and 
Science Study (TIMSS) 


| e S NATIONAL CENTER For 
EDUCATION STATISTICS 


Institute of Education Sciences 
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What is TIMSS Advanced? 


The Trends in International Mathematics and 
Science Study Advanced (TIMSS Advanced) is 
an international assessment and research project 
designed to measure advanced mathematics and 
science achievement at the twelfth grade, as well 
as collect information about school and teacher 
practices related to instruction. TIMSS Advanced 
was administered in 1995 and 2008, with the 
United States participating previously in 1995. 
In 2015, TIMSS Advanced will involve students 
from 11 countries, including the United States. 


TIMSS Advanced is sponsored by the International 
Association for the Evaluation of Educational 
Achievement (IEA) and managed in the United 
States by the National Center for Education 
Statistics (NCES), part of the U.S. Department 

of Education. 


Why is TIMSS Advanced 
important? 


TIMSS Advanced provides a unique opportunity 
to compare the achievement of U.S. twelfth-grade 
students who have taken advanced mathematics or 
physics courses with that of their peers in countries 
around the world on advanced mathematics 

and physics content. TIMSS Advanced will 
provide educational policymakers with valuable 
information about how many students are 
excelling at highly specialized science, technology, 
engineering, and mathematics (STEM) content 
in a global context. The results will inform 
national discussions about education, 
preparedness for postsecondary education, 
and international competitiveness. 


TIMSS Advanced provides valuable 
benchmark information on 

how U.S. students compare 

to students around the world, 


Exhibit B-4. TIMSS Advanced 2015 Brochure—Continued 


allows educators and policymakers 
to examine other educational 
systems for practices that could 
have application to the United 
States, and contributes to 

ongoing discussions of ways 

to improve the quality of 
education of all students. 


What type 
of assessment 


is TIMSS Advanced? 


The TIMSS Advanced mathematics and 
physics assessment is developed through 

an international consensus-building process 
involving input from U.S. and international 
experts in mathematics, science, and measurement. 
In a final step, the assessment is endorsed 
as suitable by all participating countries. 
The assessment contains a mix of questions; 
some require students to select appropriate 
responses while others require that 
students solve problems and provide 
written answers. Examples of 
released TIMSS items are available 
at http://nces.ed.gov/timss/ 
educators.asp. 


Exhibit B-5. TIMSS Advanced 2015 FAQ 


TIMSS Advanced 2015 


What are TIMSS and TIMSS Advanced? 


The Trends in International Mathematics and Science 
Study (TIMSS) is an international assessment and 
research project designed to measure trends in 
mathematics and science achievement at the fourth- 
and eighth-grade levels as well as collect information 


about school and teacher practices related to instruction. 
Since 1995, TIMSS has been administered every 4 years. 


In 2015, TIMSS will involve students from more than 
50 countries, including the United States. 


TIMSS Advanced measures trends in advanced 
mathematics and physics achievement for students 

in their final year of secondary school. It will provide 
information about how many students are excelling 
at highly specialized science, technology, engineering, 
and mathematics (STEM) content in comparison to 
students in other countries around the world. TIMSS 
Advanced was administered in 1995 and 2008 and 
will be administered again in 2015. In 2015, TIMSS 
Advanced will be administered in 11 countries, 
including the United States. 


Why was my school selected 
for participation? 


Schools of varying demographics and locations were 
randomly selected so that the overall U.S. sample is 
representative of the overall U.S. school population. 
The random selection process is important for ensuring 
that a country’s sample accurately reflects its schools 
and, therefore, can fairly be compared with samples 

of schools from other countries. 


Will all our twelfth-grade students 
be asked to participate? 


No. Only twelfth-grade students who have taken or 
are taking calculus or advanced physics courses are 
eligible to participate. A sample of eligible students 
will be selected for each subject: advanced mathematics 


and physics. 


Who conducts the TIMSS assessment? 


The entire assessment process will be conducted 

by trained staff from Westat, a research organization 
under contract with the National Center for Education 
Statistics in the U.S. Department of Education. NCES 
is authorized to conduct this study under the Education 
Sciences Reform Act of 2002 (ESRA 2002), 20 U.S. 
Code, § 9543, and approval of the U.S. Office of 
Management and Budget under OMB # 1850-0695. 


How are teacher and school 
questionnaires administered? 


The teacher and school questionnaires are administered 
online from a secure website. Teacher questionnaires 
take about 35 minutes to complete and ask teachers 
questions about their experience, available resources, 
and instructional practices. School questionnaires take 
about 30 minutes to complete and ask about school 
practices and resources. 


Do teachers need to help administer 
the assessment? 


No. Westat field staff will visit the school on the day 

of the assessment, bringing with them all the materials 
required, and they will handle the entire administration 
of the assessment. 


When will the assessment be conducted? 
‘The assessment will be conducted between March 30 
and May 29, 2015. Each school will be notified of its 
scheduled assessment date in summer 2014. A TIMSS 
representative will work with schools to identify an 
alternate date should there be a conflict on that date. 


Where will the assessment be conducted? 


The assessment will be conducted in the schools that 
are selected to participate. 
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How long does the assessment take? 


The assessment session is approximately 22 hours 
and includes the administration of the assessment, 

a brief calculator survey, a questionnaire that students 
complete, and a break. The questionnaire takes 
approximately 30 minutes to complete. 


What will happen with the collected data? 


The data from the assessment will be used to evaluate 
how the knowledge and skills of U.S. students compare 
to those of their peers in other participating countries. 
By law, the data provided by schools, staff, and students 
may be used only for statistical purposes and may 

not be disclosed or used in identifiable form for any 
other purpose except as required by law [Education 
Science Reform Act of 2002 (ESRA 2002), 20 U.S. 
Code, Section 9573]. Reports of the findings from 

the assessment will not identify participating districts, 
schools, students, or individual staff. Individual 
responses will be combined with those of other 
participants to produce summary statistics and reports. 


OMB # 1850-0695 


Are schools required by federal law 
to participate? 


No. School participation is voluntary. However, 
we hope you will participate in this study so that 
students like those in your school are accurately 
and fairly represented. 


Where can I find more information 
about TIMSS? 


Visit the TIMSS website at http://nces.ed.gov/timss or 
the TIMSS international website at http://timss.bc.edu. 


For additional information about TIMSS 2015, contact 
the TIMSS U.S. information hotline at 855-445-5604 or 
email TIMSS@westat.com. 
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Exhibit B-6. TIMSS Advanced 2015 Summary of School Activities 


Summary of 
School Activities 
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For questions about TIMSS 2015, 
contact the toll-free TIMSS information 
hotline at 855-445-5604 

or email TIMSS@westat.com 


OMB # 1850-0695 


‘ies 


NATIONAL CENTER For 
EDUCATION STATISTICS 


ADVANCED 


Trends in International 
Mathematics and 
Science Study (TIMSS) 


2015 


Summary of 
School Activities 
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Exhibit B-6. TIMSS Advanced 2015 Summary of School Activities—continued 
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SUMMARY OF SCHOOL ACTIVITIES: TIMSS ADVANCED 2015 


Principal = Designate a school coordinator 


= Confirm an assessment date convenient 
for your school 


School coordinator 


Teachers of 
eligible courses 


Selected students 


= Work with the school to confirm an 
assessment date 


= Help school coordinator with assessment 
details and logistics 


= Protect school and student confidentiality 


TIMSS representative 


= Complete a brief school questionnaire on the 
characteristics of the school, its enrollment, resources, 
policies, and learning environment (available online 
or hardcopy) 


= Share the importance of participation in the study 
with school staff 


m= Arrange assessment day location 
= Confirm date and location with TIMSS assessment staff 
m Provide course catalogue, if not available online 


= Provide current class lists and transcripts for grade 
12 students through online MyTIMSS portal 


m Notify teachers, selected students, and students’ 
parents of the study and benefit of participating 


= Work with assessment staff to identify students 
with special needs 


= Coordinate the principal's completion of the school 
questionnaire (online or hardcopy) 


= Coordinate teachers’ completion of the teacher 
questionnaire (online or hardcopy) 

a Distribute parent notification letters to parents 
of selected students 


= Complete online teacher questionnaire 


= Call the school coordinator to discuss assessment day 
location and student participation 


= Review list of selected students with school coordinator 


= Support assessment day activities 


= Encourage students fo participate and do their best 
on the assessment 


= Confirm space for assessment is problem-free 


= Collect completed school and teacher 
questionnaires (if not completed online) 
and give to assessment staff 


= Ensure all sampled students attend 
the assessment session 


= Meet with assessment staff and provide 
feedback about the assessment process 


= Attend the assessment session, complete 
the assessment and student questionnaire 


= Conduct assessment from start to finish 


= Furnish all the assessment materials, pencils, 
and test booklets 


= Conduct a brief debriefing interview with the school 
coordinator at the end of the assessment 


= Maintain security of all materials 


= Represent other similar U.S. schools 

= Receive a $200 check for the school 

= Receive a confidential report on how 
students selected from your school performed 
onTIMSS assessment questions that are 
made public 


Receive a $100 personal check 
Opportunity to participate in a professional 
development web seminar series 

on international assessments and 
educational policy 


Represent U.S. teachers in the international 
study results 

Opportunity to participate in a professional 
development web seminar series 

on international assessments and 
educational policy 


Receive a small thank-you gift 


Represent the United States in 
the international study 


Ensure quality and uniformity of data 
collected across the United States 
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TIMSS 2015: Instructions for Grade 4 Submit Class List 


TIMSS needs a complete and current list of all of your school’s fourth-grade classes in order to randomly 
select classes to be assessed. Typically, two classes are sampled in each school, and all students in the 
selected classes will be assessed. 


On www.mytimss.com, select Submit Class List from the navigation menu on the left. Enter information 
for each fourth-grade class in the web form. After you have entered information for each class, click on 
the Save button, and another row will appear for you to enter information about another class. 
Continue until you have entered information for all classes that contain fourth-grade students. 


Include the following information for each class: 


e Class Name 

e Class Group or Track (If applicable) 
e Number of Students 

e Class Exclusion Status (If applicable) 
e Name of Mathematics Teacher 

e Name of Science Teacher 


When you have entered information for fourth-grade classes in your school, click on the Finished button. 


Example 1: Small school with the same Mathematics and Science Teacher for each class 


Please refer to important definitions and examples by hovering over the @ in each column header. These definitions are also provided in the Class Roster Definitions. 


Use the Save button to save information you have entered for each class. 

If you need to delete a class, click on the @ at the beginning of a saved class row. 

Use the Print button to print a list of the classes you included for reference. 

Click on the Finished button when you have entered all fourth-grade classes in the form. 


? é @ Number @ Class z : 
‘U' Class Name \U Class Group of Exclusion \L' Name of Mathematics Teacher \U’ Name of Science Teacher 
Students Status (If applicable) 
Select. v Select. ’ 
When you choose "Submit Class List" for Grade 4, you 
will see this empty form. 
Save Print Finished 
Grade 4 


ct meaner ® class z : 
@ Class Name ® class Group Exclusion \U' Name of Mathematics Teacher \U' Name of Science Teacher 
le Status (If applicable) 


eee bd Select. Ad 


Save records the class information and opens a new line for another class 


Print Finished 


Grade 4 


mw Nenwver @ class 
Class Name ' Class Group Exclusion 
cua Status (Ifa 


Jensen isabiliti ‘Anne Jensen 


Trevor Adams Trevor Adams 


- — Choose "Finished" once you have entered all class information for 
Save Print Finished Grade 4. No further editing is possible when you choose this option. 
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Example 2: Larger school with multiple Grade 4 classes 


Grade 4 


e : @ Number ® class _ e 
\U Class Name \U' Class Group of Exclusion \L’ Name of Mathematics Teacher \U' Name of Science Teacher 
Students Status (If applicable) 


Grade 4-3 Students with intellectual disabilities ~ | William Peters 


Select. v Select. 


Julia O'Donnell 


Julia O'Donnell 


Julia O'Donnell 


"Save" records the class information and opens a new line for another class 


[i sees | Print Finished 


Grade 4 


bs y ab @® Class ks E 
{ Class Name “ Class Group Exclusion “) Name of Mathematics Teacher {) Name of Science Teacher 
Status (If applicable) 


Grade 4-1 Students with intellectual disabilities ~ ||Paul Smith Julia O'Donnell 


Grade 4-3 Students with intellectual disabilities » ||William Peters Julia O'Donnell 


Save Print 


You can print any time 


Additional tips for submitting information about your fourth-grade classes: 


e Refer to the important definitions in the Class Roster Definitions document in the Documents 
section of MyTIMSS. 


e Record the Class Name that is typically used by your school to refer to the class. 

e For classes with students from more than one grade level, include in Number of Students only 
those students in grade 4. For example, if the class has 5 third-graders, 6 fourth-graders, and 4 
fifth-graders, enter “6” as the Number of Students in this class. 

e Use the most current enrollment information. 

e Include all classes, even if they typically are excluded from your state testing program, or all 


students take alternate assessments. You can indicate information about the class in Class 
Exclusion Status. 


e Name of mathematics teacher and name of science teacher may be the same. 
e Print a list of the classes you included for your future reference using the Print button. 


If you need assistance, email TIMSSefile@westat.com or call 1-855-457-1577. 
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TIMSS 2015: Instructions for Grade 4 Submit Student List 


Along with the list of classes, TIMSS needs a complete and current list of all students in grade 4 in order 
to draw a random sample of classes (and therefore students) to participate in the assessment. Your 
student data electronic file (E-File) must be submitted as a Microsoft Excel file. 


You may use one of the TIMSS Grade 4 Excel Templates (located in Documents on www.mytimss.com 


and described below) or you may provide an Excel file with the same information. If you cannot submit 
your student data with this information in an Excel file, please call or email the TIMSS E-File Help Desk at 


TIMSSefile@westat.com or 1-855-457-1577. 
Step 1 — Compile Data in an Excel File 


Prepare an Excel file with the following data elements for all students in grade 4. 


Data Element Description 

Class The fourth-grade class the student is enrolled in. Please use the 
same class names you used in Submit Class List. 

Teacher Name Teacher of the above named class 

Student Name The preferred format is First Name, Middle Name (or Initial), and 


Last Name in separate columns. However, TIMSS will accept 
student names in one column. 


Sex Codes (numeric or text) for Male/Female 

Date of Birth The preferred format is Month of Birth and Year of Birth in 
separate columns. However, TIMSS will accept Date of Birth in 
one column. 

Student with a Disability | School-defined codes for: 

(SD) Status e Yes, IEP: Student has an Individualized Education Plan (IEP). 


Students with 504 plans that need accommodations can be 
coded this value as well. 


e No, not SD: 

e Information unavailable at this time. 
English Language School-defined codes for: 
Learner (ELL) Status e Yes, ELL 


e No, not ELL 
e Information unavailable at this time. 


You may use one of the templates provided or create your own Excel file with these data for each 
student. It is preferred that you include column headers as the first row in your E-File, as in these 
templates. However, E-Files without column headers will be accepted. 


Template 1 has student name in three separate columns, and month of birth and year of birth in two 
separate columns. 


B iS D E F G H I J 
Student Middle Month Yearof Student witha English Language 
Class Name |Teacher Name Student FirstName Name or Initial StudentLastName Sex of Birth Birth DisabilityStatus Learner Status 


a 
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Template 2 has student name in one column, and date of birth in one column. 


D E F G 


Class Name ITeacher Name Student Name Sex Date of Birth Student with a Disability Status English Language Learner Status 


Tips for ensuring the E-File process goes smoothly: 


e Use the template or provide column headers. Student information should begin on the second 
row. There should be no empty rows within the student data. 

e Be sure to give your file a unique, descriptive name. Within Excel, click on File, then Save As, 
and give your file a name such as “Your School Name Grade 4.xls.” 

e The first row of data in your file will be read as the column header unless you indicate otherwise 
on the Submit Student List webpage. Each succeeding row will be considered as a student 
record. 

e There should be only one worksheet with data in the Excel file. If there are other worksheets, 
they must not contain any data. 


Step 2 — Upload Your Excel File 


Once your file is prepared and checked for accuracy and completeness, login to www.mytimss.com and 
select “Submit Student List” from the left-hand menu. 


Ss es Once you have submitted an E-File, the file’s information 
pleased | lplnadod by will appear under these headings, as shown in the box 
below 


The e-Filing system needs to know if the 
Does your student data file contain column headers: Yes ~ first row of the file contains headings 


New file:||| Browse_ | |TIMSS Gr4 test efile one colxlsx You can type in the file name or use 
Browse. | "Browse" to find the file on your 
“ computer 
Upload |/Click here to go to the next step 


TIMSS Gr 4 test efile three col.xlsx | 13471 2/3/2015 2:00:14 PM | James Morris 


Step 3 — Identify Your Columns 


We need to know what information is in each column of your student data file (E-File). If you provided 
column headers in your E-File (preferred), they are displayed in Column Heading Is in the table. If you 
did not provide column headers, Column Heading Is will contain numbers for each column in your 
submitted E-File. Click on the down arrows in Your Column Contains to select descriptions for each 
column header. If there is no appropriate description in the drop-down list, please select N/A. 


Select Next after all columns have been identified. 
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Column Heading Is Your Column Contains 


Class Class oa 


Testis 
Student with a Disability ’ 


ELL Status Column Contains ... |r| 
Column Contains ... 
Student Name 
Student Name: First 
Student Name: Middle 
Student Name: Last 


| Previous || Next | 


You need to define the Date of Birth 
contents of the Birth Date: Month of Birth 
columns in your file by Birth Date: Year of Birth 
choosing field options Sex 
from the drop-down Student with a Disabili 
lists. English Language Learner 
Teacher Name 


Teacher Name: Last 
Teacher Name: First 
Teacher Name: Middle 


Step 4 — Match Your Values to TIMSS Codes 


TIMSS also needs to know the values for some of the columns in your E-File. The tables contain the 
following information for each column: 


e Your Values: The values in your E-File 

e TIMSS Codes: Click on the down arrow and select the TIMSS Code that best matches Your 
Values 

e Number: The number of students in your E-File with that value 

e Percentage: The percentage in your E-File with that value 


M 


‘Select Code [x] 11 52.38% 
Select Code 


N/A In this step, you explain to the system 


what the data in your columns means 
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Use the Previous and Next buttons to navigate from column header to column header. 


If there is an error in your data file, exit E-File, correct the problem in your Excel file, and upload the file 
again. 


Step 5 — Verify Your E-File 


The table in this section summarizes the information you have provided on your student list (E-File). 
Please review this summary and verify that the information is correct. Total Enrollment at the bottom 
of the table should match the number of students in your E-File, and the total number of students 
currently enrolled in grade 4. 


Record whether the information is correct or incorrect by selecting the appropriate button at the 


bottom of the page. Then click the Submit button. If you select INCORRECT, you will need to correct 
your E-File and resubmit it. 


F]tumner | rercentase | 


Date of Birth 

MDY 27 100% 
Sex 

Male 15 55.56% 


Student with a Disability 
Yes, IEP 


| 3 | 11.11% 


No, not SD 88.89% 


English Language Learner 
Yes, ELL 3 11.11% 
No, not ELL 24 88.89% 
Grade 4-1 15 55.56% 
Grade 4- 2 12 44.44% 


Total Enrollment: ) This number is the 
total number of 

- ee! students in the e- 

‘ Information is CORRECT and CURRENT. File. It should 

© Information is INCORRECT. match your 

enrollment 

numbers. 


Review your data and 
Previous || Submit | choose an option. Click 
"Submit" to finish. 


If you need assistance, please contact the TIMSS E-File Help Desk at TIMSSefile@westat.com 
or 1-855-457-1577. 
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TIMSS 2015 Grade 4: Class Roster Definitions 


e Class Name: Record the class name that is typically used by your school to refer to the class. For example, it may be that 
your school uses the grade plus a letter for the class name (4a, 4b, etc.), the grade plus a number (4.1, 4.2, etc.), the class 
location (Room 7, Room 8, etc.), or some other scheme. This is important because these identifiers will be used to indicate 
to the Test Administrator which classes will be tested. 


e Class Group or Track (if applicable): If your school assigns students to specific classes based on their ability, please indicate 
the relevant level: Low ability, Average ability, or High ability. 


e Number of Students: Enter the number of fourth-grade students in each class. In the case of multi-grade classes (e.g., 
students from more than one grade level in the same class), only the fourth-grade students should be counted as a class in 
the list. For example, if three Grade 3 students, five Grade 4 students, and ten Grade 5 students form a multi-grade class, 
then you should record five students for the number of students in this multi-grade class. 


e Class Exclusion Status (if applicable): As a rule, all classes are to be included. TIMSS 2015 will offer many accommodations 
that should allow most students to participate. Click on the Document link on the left panel to see what accommodations 
are provided or allowed. All class-level exclusions must be approved. If you indicate a class-level exclusion, a TIMSS 
representative will contact you to discuss. 


Example of potential class-level exclusions include classes where al/ students belong to at least one of the following three 
exclusion status categories, and none of the students can be assessed with TIMSS accommodations: 


1 = Students with functional disabilities (students who have physical disabilities in such a way that they cannot 
perform in the TIMSS testing situation. Students with functional disabilities who are able to perform should be 
accommodated in the test situation, within reason, rather than excluded); 


2 = Students with intellectual disabilities (students who are considered, in the professional opinion of the school 
principal or by other qualified staff members, to have severe intellectual disabilities or who have been tested as 
such. This category includes students who are emotionally or mentally unable to follow even the general 
instructions of the test. Students should not be excluded solely because of poor academic performance or normal 
disciplinary problems. It should be noted that students with dyslexia, or other such learning disabilities, should be 
accommodated in the test situation, within reason, rather than excluded; 


3 = Non-native language speakers (students who are unable to read or speak the language(s) of the test and would 
be unable to overcome the language barrier in the test situation). 


If all students in the excluded class do not belong to the same exclusion category, please identify the category 
corresponding to the majority of students. 


e Name of Mathematics Teacher: Name of the teacher teaching mathematics content to the fourth-grade class. 


e Name of Science Teacher: Name of the teacher teaching science content to the fourth-grade class. 
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TIMSS 2015: Instructions for Grade 8 Submit Class List 


TIMSS needs a complete and current list of all of your school’s eighth-grade mathematics classes in 
order to randomly select classes to be assessed. Typically, two classes are sampled in each school, and 
all students in the selected classes will be assessed. 


Step 1 -— Enter information about all eighth-grade mathematics classes 


On www.mytimss.com, select Submit Class List from the navigation menu on the left. Enter information 
for each eighth-grade mathematics class in the web form. After you have entered information for each 
mathematics class, click on the Save button, and another row will appear for you to enter information 
about another mathematics class. Continue until you have entered information for all mathematics 
classes that contain eighth-grade students. 


Include the following information for each class: 


e Class Name 

e Class Group or Track (|f applicable) 
e Number of Students 

e Class Exclusion Status (If applicable) 
e Name of Mathematics Teacher 


Example: 
Grade 8 
= = @ Number @ Class ms 
\U’ Class Name \l' Class Group of Exclusion \L’ Name of Mathematics Teacher 
Students Status (If applicable) 
| | Select.. v | Select... * | 


Save | The mathematics class list is blank the first time you choose "Submit Class List" 


Grade 8 


? os @ Number @ Class K 
\U' Class Name \U Class Group of Exclusion \U Name of Mathematics Teacher 
Students Status (If applicable) 


< | 


Charles Moore 


Intro to Algebra High ’ 24 Select... Charles Moore 
Click "Save" to save a class and create an empty row for another class. When you are finished, move on 
Save | to the science teacher section, below. 


Step 2 — Enter information about your eighth-grade science teachers 
TIMSS also needs a complete and current list of all of your school’s eighth-grade science teachers in 


order to link information about science teachers to the assessed students. Enter each teacher’s name 
and the course the teacher teaches (e.g. biology, integrated science) into the form. Click Save after 
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entering each teacher. Continue until you have entered information about all science teachers who 
teach eighth-grade students. 


Example: 


i “' Course Name “\’ Name of Science Teacher 


[ [J 


The science course and teacher 


Save 


Print Ennead list is blank at first. 


= @ Course Name @ Name of Science Teacher 


Click "Save" to save the science course and 
: — teacher and create an empty row for 
Finished another one. 


= Course Name Name of Science Teacher 


Earth Science Jess Alban 


Biology 1 Nate Halloran] 


Choose "Finished" once you have entered all mathematics 


= classes and science courses for Grade 8. No further editing is 
Finished possible when you choose this option. 


When you have entered information for eighth-grade mathematics classes, and eighth-grade science 
teachers, in your school, click on the Finished button. 


Additional tips for submitting information about your eighth-grade mathematics classes: 


Refer to the important definitions in the Class Roster Definitions document in the Documents 
section of MyTIMSS. 

Record the Class Name that is typically used by your school to refer to the class. 

For math classes with students from more than one grade level, include in Number of Students 
only those students in grade 8. For example, if the class has 5 seventh-graders, 6 eighth- 
graders, and 4 ninth-graders, enter “6” as the Number of Students in this class. 

Use the most current enrollment information. 

Include all classes, even if they typically are excluded from your state testing program, or all 
students take alternate assessments. You can indicate information about the class in Class 
Exclusion Status. 

Examples of science courses include: general science, integrated science, biology, physical 
science. Use the course names that you use in your school. 

Print a list of the mathematics classes and science teachers you included for your future 
reference using the Print button. 


If you need assistance, email TIMSSefile@westat.com or call 1-855-457-1577. 
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TIMSS 2015: Instructions for Grade 8 Submit Student List 


Along with the list of classes, TIMSS needs a complete and current list of all students in grade 8 in order 
to draw a random sample of mathematics classes (and therefore students) to participate in the 
assessment. Your student data electronic file (E-File) must be submitted as a Microsoft Excel file. 


You may use one of the TIMSS Grade 8 Excel Templates (located in Documents on www.mytimss.com 
and described below) or you may provide an Excel file with the same information. If you cannot submit 


your student data with this information in an Excel file, please call or email the TIMSS E-File Help Desk at 


TIMSSefile@westat.com or 1-855-457-1577. 
Step 1 — Compile Data in an Excel File 


Prepare an Excel file with the following data elements for all students in grade 8. 


Data Element Description 

Mathematics Class The mathematics class the student is enrolled in. Please use the 
same class names you used in Submit Class List. 

Mathematics Teacher Teacher of the above named mathematics class that the student is 
taking 

Science Teacher Name of science teacher who teaches science to each student 

Student Name The preferred format is First Name, Middle Name (or Initial), and 


Last Name in separate columns. However, TIMSS will accept 
student names in one column. 


Sex Codes (numeric or text) for Male/Female 

Date of Birth The preferred format is Month of Birth and Year of Birth in 
separate columns. However, TIMSS will accept Date of Birth in 
one column. 

Student with a Disability | School-defined codes for: 

(SD) Status e Yes, IEP: Student has an Individualized Education Plan (IEP). 


Students with 504 plans that need accommodations can be 
coded this value as well. 


e No, not SD: 

e =Information unavailable at this time. 
English Language School-defined codes for: 
Learner (ELL) Status e Yes, ELL 


e No, not ELL 
e = Information unavailable at this time. 


You may use one of the templates provided or create your own Excel file with these data for each 
student. It is preferred that you include column headers as the first row in your E-File, as in these 
templates. However, E-Files without column headers will be accepted. 


Template 1 has student name in three separate columns, and month of birth and year of birth in two 
separate columns. 


B c D 2 E G H 1 Jj K 
Student Middle Month of Yearof Studentwitha English Language 


Mathematics Class Name [Mathematics Teacher Name Student FirstName Nameor Initial Student LastName Sex Birth Birth Disability Status LearnerStatus Science Teacher Name 
2 
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Template 2 has student name in one column, and date of birth in one column. 


B Cc D E E G H 
Student witha English Language 
Mathematics Class Name [Mathematics Teacher Name Student Name Sex Date of Birth Disability Status LearnerStatus Science Teacher Name 


Tips for ensuring the E-File process goes smoothly: 


e Use the template or provide column headers. Student information should begin on the second 
row. There should be no empty rows within the student data. 

e Be sure to give your file a unique, descriptive name. Within Excel, click on File, then Save As, 
and give your file a name such as “Your School Name Grade 8.xls.” 

e The first row of data in your file will be read as the column header unless you indicate otherwise 
on the Submit Student List webpage. Each succeeding row will be considered a student record. 

e There should be only one worksheet with data in the Excel file. If there are other worksheets, 
they must not contain any data. 


Step 2 — Upload Your Excel File 


Once your file is prepared and checked for accuracy and completeness, login to www.mytimss.com and 
select “Submit Student List” from the left-hand menu. 


: =e Once you have submitted an E-File, the file’s information will 
CTT R CT's appear under these headings, as shown in the box below 
The e-Filing system needs to know if the 
Does your student data file contain column headers: Yes vy _ first row of the file contains headings 


New file:|| B: _ | ITIMSS Gr8 test efi Ixl You can type in the file name 
ae oe sh sttcisaciniioanitation or use "Browse" to find the 


file on your computer 
Upload Click here to go to the next step 


TIMSS Gr8 test efile.xlsx 13471 [2/3/2015 2:00:14 PM [3ames Morris 


Step 3 — Identify Your Columns 


We need to know what information is in each column of your student data file (E-File). If you provided 
column headers in your E-File (preferred), they are displayed in Column Heading Is in the table. If you 
did not provide column headers, Column Heading Is will contain numbers for each column in your 
submitted E-File. Click on the down arrows in Your Column Contains to select descriptions for each 
column header. If there is no appropriate description in the drop-down list, please select N/A. 


Select Next after all columns have been identified. 


U.S. TIMSS 2015 and TIMSS Advanced 1995 & 2015 Technical Report and User’s Guide B-24 


Exhibit B-11. TIMSS 2015 Grade 8 Submit Student List Tip Sheet—Continued 


Mathematics Teacher Name ’ 
Science Teacher Name ’ 
Student with a Disability ’ 


ELL Status Column Contains ... ibd) 
“Column Contains ... “ 
Previous | Next Student Name 


Student Name: First 

Student Name: Middle 

Student Name: Last 

Date of Birth 

Birth Date: Month of Birth 

Birth Date: Year of Birth 

Sex 

SaaS Student with a Disabil 
fEnglshLanguage Loomer | 
/ Mathematics Teacher Name 

Mathematics Teacher Name: Last 

Mathematics Teacher Name: First 

| Mathematics Teacher Name: Middle 

Science Teacher Name 

| Science Teacher Name: Last 

Science Teacher Name: First LJ 

Science Teacher Name: Middle 

—— nn (19SS ¥. 


m 


Step 4 — Match Your Values to TIMSS Codes 


TIMSS also needs to know the values for some of the columns in your E-File. The tables contain the 
following information for each column: 


e Your Values: The values in your E-File 

e TIMSS Codes: Click on the down arrow and select the TIMSS Code that best matches Your 
Values 

e Number: The number of students in your E-File with that value 

e Percentage: The percentage in your E-File with that value 


M 


‘Select Code [y] 11 52.38% 
Select Code 


N/A In this step, you explain to the system 


what the data in your columns means 


Female 


Use the Previous and Next buttons to navigate from column header to column header. 
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If there is an error in your data file, exit E-File, correct the problem in your Excel file, and upload the file 
again. 


Step 5 — Verify Your E-File 


The table in this section summarizes the information you have provided on your student list (E-File). 
Please review this summary and verify that the information is correct. Total Enrollment at the bottom 
of the table should match the number of students in your E-File, and the total number of students 
currently enrolled in grade 8. 


Record whether the information is correct or incorrect by selecting the appropriate button at the 
bottom of the page. Then click the Submit button. If you select INCORRECT, you will need to correct 
your E-File and resubmit it. 


P| number | percentage 


Male 15 

Female 44.44% 
Student with a Disability 

Yes, IEP fF ls 18.52% 

No, not SD 81.48% 
English Language Learner 

Yes, ELL 18.52% 
No, not ELL 81.48% 


Class 


33.33% 


Intro to Algebra 


Business Math 1 


Geometry 29.63% 


Total Enrollment: 
a 


This number is the total number 


© Information is CORRECT and CURRENT. of students in the e-File. It 
© Information is INCORRECT. soir naa — 


Review your data and 
Previous choose an option. Click 
"Submit" to finish. 


If you need assistance, please contact the TIMSS E-File Help Desk at TIMSSefile@westat.com 
or 1-855-457-1577. 
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TIMSS 2015 Grade 8: Class Roster Definitions 


e Class Name: Record the class name that is typically used by your school to refer to the mathematics class. For example, it 
may be that your school uses the grade plus a letter for the class name (8a, 8b, etc.), the grade plus a number (8.1, 8.2, 
etc.), the class location (Room 7, Room 8, etc.), or some other scheme. This is important because these identifiers will be 
used to indicate to the Test Administrator which classes will be tested. 


e Class Group or Track (if applicable): If your school assigns students to specific classes based on their ability, please indicate 
the relevant level: Low ability, Average ability, or High ability. 


e Number of Students: Enter the number of eighth-grade students in each class. In the case of multi-grade classes (e.g., 
students from more than one grade level in the same class), only the eighth-grade students should be counted as a class in 
the list. For example, if three Grade 7 students, five Grade 8 students, and ten Grade 9 students form a multi-grade class, 
then you should record five students for the number of students in this multi-grade class. 


e = Class Exclusion Status (if applicable): As a rule, all classes are to be included. TIMSS 2015 will offer many accommodations 
that should allow most students to participate. Click on the Document link on the left panel to see what accommodations 
are provided or allowed. All class-level exclusions must be approved. If you indicate a class-level exclusion, a TIMSS 
representative will contact you to discuss. 


Example of potential class-level exclusions include classes where al/ students belong to at least one of the following three 
exclusion status categories, and none of the students can be assessed with TIMSS accommodations: 


1 = Students with functional disabilities (students who have physical disabilities in such a way that they cannot 
perform in the TIMSS testing situation. Students with functional disabilities who are able to perform should be 
accommodated in the test situation, within reason, rather than excluded); 


2 = Students with intellectual disabilities (students who are considered, in the professional opinion of the school 
principal or by other qualified staff members, to have severe intellectual disabilities or who have been tested as 
such. This category includes students who are emotionally or mentally unable to follow even the general 
instructions of the test. Students should not be excluded solely because of poor academic performance or normal 
disciplinary problems. It should be noted that students with dyslexia, or other such learning disabilities, should be 
accommodated in the test situation, within reason, rather than excluded; 


3 = Non-native language speakers (students who are unable to read or speak the language(s) of the test and would 
be unable to overcome the language barrier in the test situation). 


If all students in the excluded class do not belong to the same exclusion category, please identify the category 
corresponding to the majority of students. 


e Name of Mathematics Teacher: Name of the teacher teaching mathematics content to the eighth-grade class. 
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TIMSS Advanced 2015: Instructions for Grade 12 Submit Student List 


TIMSS needs a complete and current list of all 12th-grade students who have taken or are currently 
taking eligible advanced mathematics and physics courses in order to draw a random sample of 
students to participate in the TIMSS Advanced assessment. Eligible courses for your school were sent to 
you via email. Your student data electronic file (E-File) must be submitted as a Microsoft Excel file. 


You may use one of the TIMSS Grade 12 Excel Templates (located in Documents on www.mytimss.com 
and described below) or you may provide an Excel file with the same information. If you cannot submit 
your student data with this information in an Excel file, please call or email the TIMSS E-File Help Desk at 
TIMSSefile@westat.com or 1-855-457-1577. If you call, please press 2 to access the TIMSS Advanced 
E-File Help Desk. 


Step 1 — Compile Data in an Excel File 
Prepare an Excel file with the following data elements for all students in grade 12 who have taken or are 


currently taking one or more of the eligible courses in your school. Refer to the recent email from 
TIMSS@westat.com for a list of eligible courses. 


Data Element 
Student Name 


Description 

The preferred format is First Name, Middle Name (or Initial), and 
Last Name in separate columns. However, TIMSS will accept 
student names in one column. 

School or state ID. Do not use social security numbers. 

The preferred format is Month of Birth, Day of Birth, and Year of 
Birth in separate columns. However, TIMSS will accept Date of 
Birth in one column. 

Sex Codes (numeric or text) for Male/Female 

Advanced Mathematics | Name of each eligible advanced mathematics course taken 
Courses 


Student ID 
Date of Birth 


Adv Math Grade Level 


Grade level when each advanced mathematics course was taken 


Adv Math Teacher 
Name 


Teacher name of each advanced mathematics course 


Physics Courses 


Name of each eligible physics course taken 


Physics Grade Level 


Grade level when each physics course was taken 


Physics Teacher Name 


Teacher name of each physics course 


You may use one of the templates provided or create your own Excel file with these data for each 
student. It is preferred that you include column headers as the first row in your E-File, as in these 
templates. However, E-Files without column headers will be accepted. 
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emplate 1 has student name in three separate columns, and month of birth, day of birth and year of 
birth in three separate columns. It also has “Advanced Math” and “Physics” courses in separate 
orksheets. 


Advanced Mathematics Worksheet 


A 8 c. MM © lrlefs F K 
Student ID provided by Student First Student Student Last Month Dayof Year of Grade level when Advanced Advanced Mathematics Advanced Mathematics 
1 school Name Middle Initial Name of Birth Birth Birth Sex MathematicsCourse Taken Course Title Teacher Name 


Physics Worksheet 


Cc D E F G 1 J K 
Grade level when Advanced 
Student ID provided Student First Student Student Last Month Day of Year of Advanced Physics Advanced Physics Physics Teacher 
Middle Initial Name of Birth Birth Birth Sex Course Taken Course Title Name 


emplate 2 has student name in one column, and date of birth in one column. It also has “Advanced 
ath” and “Physics” courses in separate worksheets. 


Advanced Mathematics Worksheet 


Grade level when Advanced Advanced Mathematics Advanced Mathematics 
1 Student ID provided by school Student Name Date of Birth Sex Mathematics Course Taken Course Title Teacher Name 


Physics Worksheet 


Grade level when Advanced Advanced Physics Advanced Physics Teacher 
1 Student ID provided by school Student Name Date of Birth Sex Physics Course Taken Course Title Name 


emplate 3 is just like Template 1 in that student name and date of birth are separated into three 
separate columns. It also provides multiple worksheets if there are multiple eligible Advanced Math 
and/or Physics courses that need to be provided separately. 


ultiple Advanced Mathematics Worksheets 


A B c eee (aa a I J K 
Student ID provided by Student First Student Student Last Month Day of Year of Grade level when Advanced Advanced Mathematics Advanced Mathematics 
1 school Name Middle Initial Name of Birth Birth Birth Sex MathematicsCourse Taken Course Title Teacher Name 


ultiple Physics Worksheets 


c D 1 K 
Grade level when Advanced 
Student ID provided Student First Student Student Last Month Day of Year of Advanced Physics Advanced Physics Physics Teacher 
Middle Initial Name of Birth Birth Birth Sex Course Taken Course Title Name 


A I J K L M N 
Student ID Grade level when Advanced Advanced Grade level when Advanced Advanced 
provided by Student Middle Student Last Monthof Dayof Year of Advanced Mathematics Mathematics Mathematics Advanced Physics Physics Physics Teacher 
1 school Student First Name Name or Initial Name Birth Birth Birth Sex Course Taken Course Title Teacher Name Course Taken Course Title Name 


U.S. TIMSS 2015 and TIMSS Advanced 1995 & 2015 Technical Report and User’s Guide B-29 


Exhibit B-13. TIMSS 2015 Advanced Submit Student List Tip Sheet—Continued 


Tips for ensuring the E-File process goes smoothly: 


e Use the template or provide column headers. Student information should begin on the second 
row. There should be no empty rows within the student data. 

e Be sure to give your file a unique, descriptive name. Within Excel, click on File, then Save As, 
and give your file a name such as “Your School Name Grade 12.xIs.” 

e The first row of data in your file will be read as the column header unless you indicate otherwise 
on the Submit Student List webpage. Each succeeding row will be considered a student record. 


Step 2 — Upload Your Excel File 


Once your file is prepared and checked for accuracy and completeness, login to www.mytimss.com and 
select “Submit Student List” from the left-hand menu. 


' Once you have submitted an E-File, the file's information will 
Filename | Filesize | DEER CEE EEE appear under these headings, as shown in the box below 
The e-Filing system needs to know if the 
Does your student data file contain column headers: Yes ~ first row of the file contains headings 


New file:}| Browse_ ? You can type in the file name) 
| Browse | TIMSS Gr 12 test efile one col xlsx | ce fakes "Etreones™ tes #nel tien 


file on your computer 
Click here to go to the next step 


= Gr 12 test efile. xlsx | 13471 [2/3/2015 2:00:14 PM | James Morris 


After you have submitted your student list, the webpage will still have the “New file” section below the 
table describing the uploaded Excel file. If you need to update or change your student data, you can 
upload a new file here. If your e-file is correct as submitted, then you are finished. 


If you need assistance, please contact the TIMSS E-File Help Desk at TIMSSefile@westat.com 
or 1-855-457-1577, and press 2. 
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Exhibit C-1. Test Administrator Training Agenda 


TIMSS and TIMSS Advanced - Test Administrator Training Agenda 
March 11-13, 2015 


Day 1: March 11, 2015 - 9:00 AM - 5:30 PM 


Session #1: Welcome 9:00-10:00 e = Introductions 
to TIMSS and TIMSS e Introduction to TIMSS and TIMSS 
Advanced Advanced 


e Overview 

o Purpose and goals 

o Samples: school and student 
e TA role and responsibilities 


Session #2: Assignments 10:00-10:30 e Checking school assignment materials 
e Assignment log and calendar 
e School folder: previous contact, school 
assignment, Student Tracking Form, 
Teacher Tracking Form. 


10:30 - 10:45 Morning Break 


Session #3: MyTIMSS SCS 10:45-12:00 e Overview of MyTIMSS SCS 


12:00 - 1:00 Lunch 


Session #4: Test Materials 1:00-1:45 e Session box content review 

Demo e Receive sample session box 
Session #5: Administering 1:45-2:30 e Accommodations overview 

TIMSS Accommodations e Accommodations allowed in TIMSS 


e TIMSS exclusions 


Session #6: Working with 2:30-3:00 e Review procedures for working with 
Private Schools and Grade 12 private schools 
Strategies e Review of strategies to obtain grade 


12 student participation 


3:00 - 3:15 Afternoon Break 


Session #7: Preassessment 3:15-3:45 e Verify Student Tracking Form 
Activities e Check School and Teacher 
Questionnaire status 
e Preassessment call 


Session #8: Exercise #1: 3:45-5:30 e Role plays 1 and 2 
Preassessment Call Role Play 
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TIMSS and TIMSS Advanced - Test Administrator Training Agenda 


March 11-13, 2015 


Day 2: March 12, 2015 - 9:00 AM - 5:30 PM 


Session #9: MyTI MSS SCS - 
Enter Information Collected 
in Role Plays 1 and 2 


Session #10: Preparing for 
Assessment Day 


9:00- 10:00 e 


10:00-10:30 ° 


Enter information in the MyTIMSS 
SCS 


Arrive at the school 


Prepare the assessment room 


10:30 - 10:45 Morning Break 


Session #11: Booklet 
Preparation 


Session #12: Exercise #2: 
Preparing the Booklets 


10:45-11:15 e 


11:15-12:00 ° 


Prepare assessment booklets 


Prepare assessment booklets and QC 


12:00 - 1:00 Lunch 


Session #12 Exercise #2: 
Preparing the Booklets - 
continued 


Session #13: Conducting the 
Assessment and Session 
Script Review 


Session #14: Completing the 
Test Administration Form 


1:00-1:45 e 
1:45-2:45 ° 
2:45-3:00 e 


Exercise 2 continued 
Review of exercise 


Administer the assessment 
following TIMSS procedures 
Complete the Test Administration 
Form 

Review session script and timing 


Review of the TIMSS and TIMSS 
Advanced Test Administration 
Forms 


3:00 - 3:15 Afternoon Break 


Session #15: Student 3:15-4:00 e 
Questionnaire QxQs e 
Session #16: Supervising the 4:00-4:30 e 
Session 

Session #17: Concluding the 4:30-5:30 e 


Session 
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How to answer student questions 


Monitoring the session 


Record student response rate 
Calculate make-up sessions 
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Exhibit C-1. Test Administrator Training Agenda—Continued 


TIMSS and TIMSS Advanced - Test Administrator Training Agenda 
March 11-13, 2015 


Day 3: March 13, 2015 - 9:00 AM - 12:00 PM 


Session #18: Collecting 9:00-9:45 e Teacher Tracking Form 
Questionnaires and Finalizing e Packing the session box 
the Assessment e Entering status and assessment 


detail into the SCS 


Session #19: AA Training 9:45-10:30 e Review AA training materials and 
training content 


10:30 - 10:45 Morning Break 


Session #20: Privacy and 10:45-11:00 e Review assessment booklet security 
Security procedures 
Meet with Field Manager 11:00-12:00 e Review assignment and schedules 


e Discuss weekly report calls 
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Exhibit C-2. Preassessment Call Log/ROC 


TIMSS 2015 Main Study 
Preassessment Call Log/ROC 


School Name and ID: 
School Coordinator: 
School Coordinator Email: 
School Phone Number: 


Date of Assessment: 


Before contacting the school coordinator confirm that all forms are in the school folder: 
Student Teacher Linkage Form(s) [_] 
Student Tracking Form(s) [] 
Teacher Tracking Form (s) [] 


Whenever possible begin contacting the school coordinator at least two weeks prior to 
the assessment. 


Hello, this is (your name) representing the Trends in International 
Mathematics and Science Study, or TIMSS. You were contacted by the Westat home office 
earlier this year to complete the sampling of the students who will be participating in TIMSS. | 
am calling to confirm the assessment logistics, discuss accommodations, and to answer any 
questions you may have. 


1. IF WE DO NOT HAVE THE COMPLETED STUDENT TEACHER LINKAGE FORM: 


Did you receive the email alerting you that the Student Teacher Linkage Form(s) are ready for 
you in MyTIMSS? 


L] Yes (ASK THE SCHOOL COORDINATOR TO LOCATE THE FORM BACK AND 
CONTINUE) 


There are two forms, one form for each assessment session to be conducted at your school. On 
each form we need you to record the student’s gender and date of birth. If any of the sampled 
students have special needs and should be excluded, please put the appropriate exclusion code 
in the Exclusion Status column. We will discuss any student who can be assessed with 


accommodations allowed by TIMSS shortly. 


We also need you to list each sampled student’s math and science teachers in the appropriate 


columns. 


Do you have any questions about this form? 
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Exhibit C-2. Preassessment Call Log/ROC—Continued 


TIMSS 2015 Main Study 
Preassessment Call Log/ROC 


ANSWER ANY QUESTIONS. IF YOU DO NOT KNOW THE ANSWER TO A QUESTION TELL 
THE SCHOOL COORDINATOR THAT YOU WILL GET BACK TO HIM/HER. 


L] No (IF NO, LET THE SCHOOL COORDINATOR KNOW THAT AN EMAIL WILL BE 
SENT AND CONTINUE WITH THE CALL) 


2. Review the Student Tracking Form (STF) to confirm demographic data for each sampled 
student, as well as to confirm any student exclusion. Update the form as necessary. 


IF STUDENT IDS (RATHER THAN STUDENT NAMES) WERE USED ON THE STUDENT 
TRACKING FORM, REMIND THE SC THAT THEIR COPY OF THE STUDENT TRACKING 
FORM (WHICH INCLUDES STUDENT NAMES) SHOULD BE READILY AVAILABLE WHEN 
YOU ARRIVE AT THE SCHOOL. 


3. Review the school and teacher questionnaire status. Record the status on the Teacher 
Tracking Form. 


USE THE SCHOOL ADMINISTRATOR/TEACHER QUESTIONNAIRE STATUS REPORT 
FROM THE SCS TO DETERMINE IF THERE ARE QUESTIONNAIRES THAT ARE NOT 
COMPLETED. IF THERE ARE MISSING QUESTIONNAIRES, TELL THE SC TO REMIND 
THE TEACHERS TO COMPLETE THEM. IF THE SC ASKS FOR HARDCOPY 
QUESTIONNAIRES EMAIL THE HOME OFFICE AT TIMSS@WESTAT.COM REQUESTING 
THE NUMBER AND TYPE OF QUESTIONNAIRES THAT SHOULD BE SENT. WHEN 
REQUESTING HARD COPY, PLEASE PROVIDE THE FOLLOWING: 


FOR THE SCHOOL QUESTIONNAIRE: 


e SCHOOL ID 
e SCHOOL NAME 
e SCHOOL COORDINATOR NAME 


FOR TEACHER QUESTIONNAIRE: 


SCHOOL ID 

SCHOOL NAME 

SCHOOL COORDINATOR NAME 
TEACHER ID 

TEACHER NAME 

SESSION ID 

SESSION NAME 

SUBJECT 
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Exhibit C-2. Preassessment Call Log/ROC—Continued 


TIMSS 2015 Main Study 
Preassessment Call Log/ROC 


4. Now|! would like to verify the assessment date for the assessment. 


My records show that we will be conducting the assessment on 
date). 


DISCUSS ASSESSMENT DATE. 


(assessment 


5. Let’s review the students who can be assessed with TIMSS allowed accommodations. 


Go over each student identified as needing accommodation on the Student 
Tracking Form. Record the student information on the Accommodation Planning 
Form and mark the appropriate accommodation to be provided to the student. 
Answer any questions the school coordinator has about the accommodations. 


Refer to the Accommodation Chart. 


DISCUSS ACCOMMODATIONS. 


6. Confirm and/or set time and location for each assessment session. 


Session Number Time Location 


Administrator 
TAIAA/School Staff 


DISCUSS TIMING OF SESSIONS. 
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Exhibit C-2. Preassessment Call Log/ROC—Continued 


TIMSS 2015 Main Study 
Preassessment Call Log/ROC 


7. Discuss procedures for gathering the students on assessment day. 


8. How do you plan to notify the selected students of the assessment? 


9. Discuss grade 12 participation. 


10. Discuss calculator use. 


11. Review parent notification procedures and receipt of consent forms if applicable. If consent is 
needed in the school, discuss with the SC what can be done to obtain consent. 


YOU MUST COLLECT A DATED COPY OF THE LETTER SENT TO THE PARENTS ON 
ASSESSMENT DAY. 
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Exhibit C-2. Preassessment Call Log/ROC—Continued 


TIMSS 2015 Main Study 
Preassessment Call Log/ROC 


12. Make arrangements for the SC, school principal, or classroom teacher to be available in each 
session to introduce you to the class. 


13. Review the handling of emergencies, problem situations, bathroom breaks and procedures 
for dismissing students at the end of the session. 


14. Ask for directions to the school, if you need them. 


15. Ask where you should report when you arrive at the school and tell the School Coordinator 
that you will be arriving at least one hour before the scheduled assessment time to get your 
materials ready. 


16. Who should the school honorarium check be made out to? 
CONFIRM SPELLING OF THE NAME (SCHOOL, PRINCIPAL, ETC.). 


17. For school coordinator honorarium, confirm the spelling of the name. 


18. THANK THE SCHOOL COORDINATOR. 
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Exhibit C-2.  Preassessment Call Log/ROC—Continued 


TIMSS 2015 Main Study 
Preassessment Call Log/ROC 


RECORD EACH CALL 


Record of Contacts: 


Person you spoke to Date Time 


Outcome 
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Exhibit C-3. TIMSS Fourth-Grade Session Script 


t a TRENDS IN INTERNATIONAL 
MATHEMATICS AND SCIENCE STUDY 2015 
SESSION SCRIPT: GRADE 4 


Administering the Grade 4 TIMSS Assessment 


The instructions marked with the LJ symbol and printed in bold in the administration script must 
be read aloud to the students word for word to ensure that the testing sessions are conducted in the 
same way in all countries. Although you should become familiar with these instructions before the 
actual testing, do not attempt to memorize them. Read these instructions exactly as they ate written. 
Comments that are not in bold are not to be read aloud. They are instructions for you only. 
To begin the testing session: 

- Make sure that the Class ID is recorded at the top of the Test Administration Form. If 

missing, you can find it on the Student Tracking Form, column [b]. 


- Make sure that students are seated quietly, with nothing on their desks except for a 
pencil. 


- Record the current time in cell Qa) of the Test Administration Form. 


- Begin reading the administration script. 
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Exhibit C-3. TIMSS Fourth-Grade Session Script—Continued 


Introduction 


ian Good morning/afternoon, everyone! My name is (YOUR NAME). This school has 
been chosen to take part in an important international project to study what students 
around the world know and can do in mathematics and science. Different countries 
from all over the world are taking part in this study. You will be taking a 
mathematics and science test. While I talk to you about today’s test, I would like you 
all to be quiet, stay at your desks, and listen carefully. 


ian If you still have any schoolbooks, papers, or other materials—for example, a ruler— 
on your desk, please put them away. All electronic devices, such as calculators, cell 
phones, tablets, portable computers, and photo or video cameras, must be turned off 
and stored away for the duration of the test administration. 


LO Now I will pass out the test booklets. Do not remove the booklets from the envelopes 
until I tell you to do so. Each of you will receive one test booklet. Not all booklets are 
the same. 


Students should have only a pencil and his or her booklet for the duration of the achievement test 
administration. 


As you hand out the test booklets, make sure that each student receives the booklet specially 
prepared for him or her. You can do that by reading the name on each test booklet and giving it to 
that student. Do not allow students to open the test booklets until you tell them to do so. 


Record the student’s participation status in column 7 (Achievement Session) of the Student Tracking 
Form. 


If you are administering a makeup session, use the shaded part of column 7 (Achievement Session). 


If a student is absent, put that test booklet aside. Do not give it to anyone else, since each test 
booklet is marked for a specific student. 


If there is a student in the classroom who is not listed on the Student Tracking Form, or an 
originally assigned booklet is damaged, assign one of the three spare booklets or one from your bulk 
supplies to the student. 


After the test booklets are passed out and the Student Tracking Form has been completed, say the 
following to the students: 


aa) Does everybody have his or her test booklet? 
If yes, then continue. If not, find out why and proceed as described before. 


ian Please take your test booklet out of the envelope. Turn to the first page in the booklet 
that says “General Directions.” Please read along as I read the directions aloud. 


Please do not admit any students into the testing session at this point. 
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Exhibit C-3. TIMSS Fourth-Grade Session Script—Continued 


General Directions 


LL) = In this test, you will answer questions in mathematics and science. You may find 
some questions easy, and you may find some questions difficult. Try to answer all 
questions, the difficult ones as well as the easy ones. 


ian For some questions, you choose the answer you think is correct, and fill in the oval 
next to it. Example 1 shows this kind of question with the oval next to the correct 
answer filled in. 


How many minutes are there in 1 hour? 


® 12 


24 
© «0 
(0) 120 
LO The oval with the letter “C” has been filled in because there are 60 minutes in one 


hour. If you are not sure about the answer to a question, fill in the oval next to the 
answer you think is best, and move on to the next question. 


ina) If you decide to change an answer to a question, completely erase your first choice 
and then fill in the oval next to your new choice. 
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Exhibit C-3. TIMSS Fourth-Grade Session Script—Continued 


(aa) 


(aa) 


(aa) 


For some questions, you must write your answer in the space below the question. 
Words, drawings, or numbers are used in answers to these questions. Example 2 
shows a question like this. 


There is a flag in John’s garden. Sometimes it hangs from the flag pole and 


sometimes it waves as shown below 


What makes the flag wavef 


~ 4 fy 
The wind makes therxia ) Wave. 


Sometimes to get full points, you will need to explain your answers for science 
questions ot show your work for mathematics questions. Keep your writing and 
calculations as neat as possible, and give mathematics answers in their simplest 
form. For problems in the test involving money, suppose you are in a country where 
the “Zed” is used as the unit of money, like dollars. 


When you are asked to write an answer, be sure that your handwriting is clear. Think 
carefully about each question, and answer as completely as possible. If you are not 
sure about the answer, give the answer you think is best, and move on to the next 
question. 


You will have 36 minutes to work in the first part of your test booklet. You will then 
have a short break. After the break, you will work for another 36 minutes in the 
second part. 


You may not use a calculator during the test. 


Make sure students with the calculator accommodation know they are allowed to use their 
own calculator during the test. 
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Exhibit C-3. TIMSS Fourth-Grade Session Script—Continued 


Instructions for Part 1 


ian Read each question carefully, and answer it as well as you can. If you are not sure 
about the answer to a question, choose or write the answer you think is best, and 
continue with the next question on the test. 


ina You will have 36 minutes to complete this part of the test. 
ina Do not start working until you are told to do so. 
ima At the end of the first part of your booklet, you will see a STOP message. Do not 


continue with the rest of the booklet until you are told to do so. If you have 
completed Part 1 before the time is up, you can review your answers until the time is 
up or take a book you have with you and read quietly at your desk. 


ae Do you have any questions? 


When all problems are resolved and you have the students! attention again, record the current time 
in cell Ob) of the Test Administration Form. Then say: 


CQ Turn the page to the beginning of Part 1, and start working immediately. Remember, 
you have 36 minutes to complete the first part of the booklet. 


Begin timing the 36 minutes for Part 1. Record the current time in cell (10a) of the Test 
Administration Form. 


Make sure all students begin working on the correct part of the booklet. Remember that you are not 
allowed to help the students with the test. While the students are working, you should move around 
the room to see that students are working on the correct section of their booklets. 


Ten minutes before the testing session ends, say the following: 


ian You have 10 minutes left before the break. Make sure you try to finish answering all 
of the questions in the first part of your booklet before the break. 


After the last 10 minutes have passed, say: 


ima Your time is up. Please close your booklets, and put your pencils down. Do not write 
anything more. We will now take a short break. 


Record the current time in cell (10b) of the Test Administration Form. 


Break 


If the room will be left unattended during the break, collect the booklets from the students one by 
one. Keep the booklets secure during the break time. You will then redistribute the booklets after 
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Exhibit C-3. TIMSS Fourth-Grade Session Script—Continued 


the break, just like you did at the beginning of the testing session, making sure that each student 
receives the same test booklet he/she was working on during the first half of the testing session. 
Instructions for Part 2 

Record the current time in cell (11a) of the Test Administration Form. 


Make sure all the students are seated. When the students are seated and quiet redistribute the test 
booklets, if necessary. Then, say the following: 


LO Does everybody have his or her test booklet? 


When all problems are resolved and you have the students’ attention again, proceed with the 
instructions for Part 2. 


LX) = In Part 2, you will answer more questions in mathematics or science. You will have 
36 minutes to work on this part of the test. 


ina Read each question carefully, and answer it as well as you can. If you are not sure 
about the answer to a question, choose or write the answer you think is best, and 
move on to the next question. 


ima Do not start working until you are told to do so. 
ima At the end of the second part of your booklet, you will see a STOP message. If you 


have completed Part 2 before the time is up, you can review your work on the second 
part of your booklet or take a book you have with you and read quietly at your desk. 


ae Do you have any questions? 


When all problems are resolved and you have the students' attention again, record the current time 
in cell (11b) of the Test Administration Form and say: 


CQ Turn the page to the beginning of Part 2, and start working immediately. You have 
36 minutes to complete the second part of the booklet. 


Begin timing the 36 minutes for the second part of the session. Record the current time in cell (12a) 
of the Test Administration Form. 


Remember that you are not allowed to help the students with the test. While the students are 
working, you should move around the room to see that students are working on the correct section 
of their booklets. If a student is finished early, make sure that he or she has a book to read. 


Ten minutes before the testing session ends, say the following: 


ina You have 10 minutes left to work on the second part of the booklet. Make sure you 
try to finish answering all of the questions in the second part of your booklet. 


After the last 10 minutes have passed, say: 


[J Your time is up. Please close your booklets, and put your pencils down. Do not write 
anything more. 
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Record the current time in cell (12b) of the Test Administration Form, then say: 


We will now take a short break. Afterwards, I will ask you to answer a short 
questionnaire. If you leave the room, please be back on time. 


Break 


If the room will be left unattended during the break, collect the booklets from the students one by 
one. Keep the booklets secure during the break time. You will then redistribute the booklets after 
the break, just like you did at the beginning of the testing session, making sure that each student 
receives the same test booklet he/she was working on during the first half of the testing session. 


Administering the Student Questionnaire 


Before you begin the questionnaire administration: 


- Make sure that you have the corresponding Student Tracking Form and Test 
Administration Form. 


- Make sure all the students are in the class; all are seated quietly, and have a pencil. 
When ready, say: 


ian Now you will complete a short questionnaire. Turn to the colored divider page in 
your booklet. Do not begin the questionnaire until I tell you to do so. 


(J The directions are printed at the beginning of your questionnaire. I will also read 
them to you. It is important that you follow the directions very carefully so that you 
understand how to mark your answers. Now turn to the first page titled “Directions.” 


Make sure that the students have their booklets open to the Directions page before proceeding. 


ina Please follow the directions in your booklet as I read them aloud. 
Directions 


O In this booklet, you will find questions about you and what you think. For each 
question, you should choose the answer you think is best. 


(2 Let us take a few minutes to practice the kinds of questions you will answer in this 
booklet. 


GO Example 1 is one kind of question you will find in this booklet. 


Make sure that all students are following along and are looking at Example 1 in their booklets. 
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Exhibit C-3. TIMSS Fourth-Grade Session Script—Continued 


Example 1 


Do you go to school? 


Fill in one oval only. 


Yes -- © 
No o>) 
ina In Example 1, the question asks, “Do you go to school?” Below this question are a 
“Yes” and a “No”. Since you all go to school, you should all fill in the oval next to 


“Yes.” 


Give students time to fill in the oval next to “Yes” and make sure they understand how to do it. 
Once everyone has completed the example, move on to Example 2. 


Make sure that all students are following along and are looking at Example 2 in their booklets. 


CQ Example 2 is another kind of question you will find in this booklet. 


Example 2 


How often do you do these things? 


Fill in only one oval for each row. 


Every day Once or Once or Never or 
or almost twice a twice a almost 
every day week month never 

a) I talk with my friends --------------- op) @ @ @® 

b) I play sports ---------------------------- @ @ ® ® 

c) I ride a skateboard ------------------- @ re) ® @® 


(J This question asks “How often do you do these things?” Letter (a) says, “I talk with 
my friends.” You are given four choices for how often you do this: Every day or 
almost every day; Once or twice a week; Once or twice a month; and Never or almost 


never. 


B 


Fill in the oval below your answer. For example, if you talk to your friends every day 
ot almost every day, fill in the first oval under “Every day or almost every day.” 
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Give students time to fill in their answers to all parts of the Example 2 question and make sure they 
understand how to answer this kind of question. Once everyone has completed the example, move 


on to Example 3. 


Make sure that all students are following along and are looking at Example 3 in their booklets. 


(an) 


Example 3 


Example 3 is another kind of question you will find in this booklet. 


What do you think? Tell how much you agree with these 
statements. 


Fill in only one oval for each row. 


Agree Agree Disagree Disagree 
a lot a little a little a lot 

a) Watching movies is fun --------------- @D @ @ ® 

b) I lke eating ice cream D ® ® @® 

c) I donot like waking up early >) >) e>) @® 

d) Lenjoy doing chores -------------------- <>) ® ® @ 


Example 3 says, “What do you think? Tell how much you agree with these 
statements.” Letter (a) says, “Watching movies is fun.” You are given four choices 
for how much you agree with the statement: Agree a lot, Agree a little, Disagree a 
little, or Disagree a lot. 


Fill in the oval below your answer. For example, if you really agree a lot with that, fill 
in the first oval under “Agree a lot.” If you really disagree a lot, fill in the oval under 
“Disagree a lot.” 


Give students time to fill in their answers to all parts of the Example 3 question and make sure they 
understand how to answer this kind of question. Then continue reading the final directions: 


Read each question carefully, and pick the answer you think is best. 
Fill in the oval next to or under your answer. 


If you decide to change your answer, completely erase your first choice. Then, fill in 
the oval next to or under your new answer. 


Ask for help if you do not understand something or are not sure how to answer. 


Are there any questions before we start? 


If there are questions try to answer them the best you can. If there are no more questions, record the 
current time in cell (13a) of the Test Administration Form and proceed with the administration of 


the questionnaire. 
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() Turn the page to the first question and begin answering this questionnaire. You will 
have 30 minutes to answer these questions. 


After 30 minutes are up, say: 
ima Please raise your hand if you have finished answering the questions. 
If not all of the students raise their hands, allow for additional time and say: 


LQ = =You will have mote time to continue answering this questionnaite. If you have 
already finished all the questions, then you can use this time to review your answers. 
Once you have finished, please close your booklet and read quietly at your desk. 


Once all students have finished and have closed their booklets record the current time in cell (13b) 
of the Test Administration Form. Then say: 


ina Thank you very much for participating in this study. Your work will help us to learn 
more about our students and schools. 


ima Please close your booklets and insert the booklet into the envelope it was in when 
you received it and seal the envelope closed. Once you have done this, remove the 
label that contains your name from the envelope. Peel it off and I will collect it. 


Collect the labels tear them up and throw them away. Then, collect the test booklets and envelopes 
and keep them secure. Check against the Student Tracking Form to make sure that you have 
received all of them. Use the sequential number you assigned to the front of the envelope to help in 
this process. 


Thank you again for your help in conducting this important 
international study. 


Please answer questions 14 through 18 on the Test Administration Form. 
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Exhibit C-4. TIMSS Eighth-Grade Session Script 


t a TRENDS IN INTERNATIONAL 
MATHEMATICS AND SCIENCE STUDY 2015 
SESSION SCRIPT: GRADE 8 


Administering the Grade 8 TIMSS Assessment 


The instructions marked with the LJ symbol and printed in bold in the administration script must 
be read aloud to the students word for word to ensure that the testing sessions are conducted in the 
same way in all countries. Although you should become familiar with these instructions before the 
actual testing, do not attempt to memorize them. Read these instructions exactly as they ate written. 
Comments that are not in bold are not to be read aloud. They are instructions for you only. 


To begin the testing session: 
- Make sure that the Class ID is recorded at the top of the Test Administration Form. If 
missing, you can find it on the Student Tracking Form, column [b]. 


- Make sure that students are seated quietly, with nothing on their desks except for a 
pencil and their own calculator. 


- Record the current time in cell Qa) of the Test Administration Form. 


- Begin reading the administration script. 
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Introduction 


ian Good morning/afternoon, everyone! My name is (YOUR NAME). This school has 
been chosen to take part in an important international project to study what students 
around the world know and can do in mathematics and science. Different countries 
from all over the world are taking part in this study. You will be taking a 
mathematics and science test. While I talk to you about today’s test, I would like you 
all to be quiet, stay at your desks, and listen carefully. 


ian If you still have any schoolbooks, papers, or other materials—for example, a ruler— 
on your desk, please put them away. If you have a calculator, it can remain on your 
desk. However, you may not use any device that can connect to the Internet. All 
other electronic devices, such as cell phones, tablets, portable computers, and photo 
ot video cameras, must be turned off and stored away for the duration of the test 
administration. 


LO Now I will pass out the test booklets. Do not remove the booklet from the envelope 
until I tell you to do so. Each of you will receive one test booklet. Not all booklets are 
the same. 


Students should have only a pencil, a calculator, and his or her booklet for the duration of test 
administration. 


As you hand out the test booklets, make sure that each student receives the booklet specially 
prepared for him or her. You can do that by reading the name on each test booklet, and giving it to 
that student. Do not allow students to open the test booklets until you tell them to do so. 


Record the student’s participation status in column 7 (Achievement Session) of the Student Tracking 
Form. 


If you are administering a makeup session, use the shaded part of column 7 (Achievement Session). 


If a student is absent, put that test booklet aside. Do not give it to anyone else, since each test 
booklet is marked for a specific student. 


If there is a student in the classroom who is not listed on the Student Tracking Form, or an 
originally assigned booklet is damaged, assign one of the three spare booklets or one from your bulk 
supplies to the student. 


After the test booklets are passed out and the Student Tracking Form has been completed, say the 
following to the students: 


ma) Does everybody have his or her test booklet? 
If yes, then continue. If not, find out why and proceed as described before. 


ian Please take your test booklet out of the envelope. Turn to the first page in the booklet 
that says “General Directions.” Please read along as I read the directions aloud. 


Please do not admit any students into the testing session at this point. 
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General Directions 


LL) = In this test, you will answer questions in mathematics and science. You may find 
some questions easy, and you may find some questions difficult. Try to answer all 
questions, the difficult ones as well as the easy ones. 


ian For some questions, you choose the answer you think is correct, and fill in the oval 
next to it. Example 1 shows this kind of question with the oval next to the correct 
answer filled in. 


How many minutes are there in 1 hour? 


® 12 


24 
© 60 
©) 120 
ae The oval with the letter “C” has been filled in because there are 60 minutes in one 


hour. If you are not sure about the answer to a question, fill in the oval next to the 
answer you think is best, and move on to the next question. 


ina If you decide to change an answer to a question, completely erase your first choice 
and then fill in the oval next to your new choice. 
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(aa) 


(an) 


For some questions, you must write your answer in the space below the question. 
Words, drawings, or numbers are used in answers to these questions. Example 2 
shows a question like this. 


There is a flag in John’s garden. Sometimes it hangs from the flag pole and 


sometimes if Waves aS shown below 


4 


' | ‘a ry 
The wid makes the +1 AQ 


LIAVE. 


Sometimes, to get full points, you will need to explain your answers for science 
questions ot show your work for mathematics questions. Keep your writing and 
calculations as neat as possible, and give mathematics answers in their simplest 
form. For problems in the test involving money, suppose you are in a country where 
the “Zed” is used as the unit of money, like dollars. 


When you are asked to write an answer, be sure that your handwriting is clear. Think 
carefully about each question, and answer as completely as possible. If you are not 
sure about the answer, give the answer you think is best, and move on to the next 
question. 


You will have 45 minutes to work in the first part of your test booklet. You will then 
have a short break. After the break, you will work for another 45 minutes in the 
second part. 


All questions can be answered without using a calculator, but if you have one 
available you may use it. 
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Instructions for Part 1 


ian Read each question carefully, and answer it as well as you can. If you are not sure 
about the answer to a question, choose or write the answer you think is best, and 
continue with the next question on the test. 


ian You will have 45 minutes to complete this part of the test. 
ina Do not start working until you are told to do so. 
ina At the end of the first part of your booklet, you will see a STOP message. Do not 


continue with the rest of the booklet until you are told to do so. If you have 
completed Part 1 before the time is up, you can review your answers until the time is 
up or take a book you have with you and read quietly at your desk. 


ae Do you have any questions? 


When all problems are resolved and you have the students! attention again, record the current time 
in cell Ob) of the Test Administration Form. Then say: 


CQ Turn the page to the beginning of Part 1, and start working immediately. Remember, 
you have 45 minutes to complete the first part of the booklet. 


Begin timing the 45 minutes for Part 1. Record the current time in cell (10a) of the Test 
Administration Form. 


Make sure all students begin working on the correct part of the booklet. Remember that you are not 
allowed to help the students with the test. While the students are working, you should move around 
the room to see that students are working on the correct section of their booklets. 


Ten minutes before the testing session ends, say the following: 


ina You have 10 minutes left before the break. Make sure you try to finish answering all 
of the questions in the first part of your booklet before the break. 


After the last 10 minutes have passed, say: 


[LJ Your time is up. Please close your booklets, and put your pencils down. Do not write 
anything more. We will now take a short break. 


Record the current time in cell (10b) of the Test Administration Form. 


Break 


If the room will be left unattended during the break, collect the booklets in the envelopes from the 
students one by one. Keep the booklets secure during the break time. You will then redistribute the 
booklets after the break, just like you did at the beginning of the testing session, making sure that 
each student receives the same test booklet he/she was working on during the first half of the 
testing session. 


Instructions for Part 2 


Record the current time in cell (11a) of the Test Administration Form. 
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Make sure all the students are seated. When the students are seated and quiet redistribute the test 
booklets, if necessary. Then, say the following: 


(ae) Does everybody have his or her test booklet? 


When all problems are resolved and you have the students’ attention again, proceed with the 
instructions for Part 2. 


[4] = In Part 2, you will answer more questions in mathematics or science. You will have 
45 minutes to work on this part of the test. 


ian Read each question carefully, and answer it as well as you can. If you are not sure 
about the answer to a question, choose or write the answer you think is best, and 
move on to the next question. 


ian Do not start working until you are told to do so. 
ima At the end of the second part of your booklet, you will see a short calculator survey 


about using a calculator on this test and a STOP message. If you have completed 
Part 2 before the time is up, you can review your work on the second part of your 
booklet or take a book you have with you and read quietly at your desk. 


ae Do you have any questions? 


When all problems are resolved and you have the students' attention again, record the current time 
in cell (11b) of the Test Administration Form and say: 


(Q Turn the page to the beginning of Part 2, and start working immediately. You have 
45 minutes to complete the second part of the booklet. 


Begin timing the 45 minutes for the second part of the session. Record the current time in cell (12a) 
of the Test Administration Form. 


Remember that you are not allowed to help the students with the test. While the students are 
working, you should move around the room to see that students are working on the correct section 
of their booklets. If a student is finished early, make sure that he or she has a book to read. 


Ten minutes before the testing session ends, say the following: 


ima You have 10 minutes left to work on the second part of the booklet. Make sure you 
try to finish answering all of the questions in the second part of your booklet. 


After the last 10 minutes have passed, say: 


ian Your time is up. Please answer the calculator survey if you have not done that yet, 
and then close your booklets and put your pencils down. Do not write anything 
more. 


Record the current time in cell (12b) of the Test Administration Form, then say: 


We will now take a short break. Afterwards, I will ask you to answer a short 
questionnaire. Please be back on time. 
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If the room will be left unattended during the break, collect the booklets in the envelopes from the 


students one by one. Keep the booklets secure during the break time. You will then redistribute the 


booklet 


s after the break, just like you did at the beginning of the testing session, making sure that 


each student receives the same test booklet he/she was working on during the first half of the 


testing session 


Admi 


nistering the Student Questionnaire 


Before you begin the questionnaire administration: 


- Make sure that you have the corresponding Student Tracking Form and Test 
Administration Form. 


- Make sure all the students are in the class; all are seated quietly, and have a pencil. 


When ready, say: 


(aa) 


(an) 


Now you will complete a short questionnaire. Turn to the colored divider page in 
your booklet. Do not begin the questionnaire until I tell you to do so. 


The directions are printed at the beginning of your questionnaire. I will also read 
them to you. It is important that you follow the directions very carefully so that you 
understand how to mark your answers. Now turn to the first page titled “Directions.” 


Make sure that the students have their questionnaires open to the Directions page before 


proceeding. 
ian Please follow the directions in your booklet as I read them aloud. 


Direct 
(aa 


ima 


ions 
In this booklet, you will find questions about yourself. Some questions ask for facts 


while other questions ask for your opinion. 


Each question is followed by a number of answers. Shade in the oval next to or under 
the answer of your choice as shown in Examples 1, 2, and 3. 


Make sure that the students are following along and are looking at Example 1 in their booklets. 
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Example 1 


Do you go to school? 


Fill in one oval only. 


Yes 
No -- @ 
CO In Example 1, the question asks, “Do you go to school?” Below this question are a 


“Yes” and a “No.” Since you all go to school, the oval next to “Yes” is filled in. 
y' g ’ 


Example 2 


How often do you do these things? 


Fill in only one oval for each row. 


Every day Once or Once or Never or 
or almost twice a twice a almost 
every day week month never 

a) I talk with my friends --------------- @ e) @® 

b) I play sports ---------------------------- oO) ) ® 

c) Iride a skateboard ------------------- @O @ >) 


CO In Example 2, the question asks “How often do you do these things?” Letter (a) 
says, “I talk with my friends.” You are given four choices for how often you do this: 
Every day or almost every day; Once or twice a week; Once or twice a month; and 
Never or almost never. 


E 


In this example, the student talks to his or her friends every day or almost every day 
and the first oval under “Every day or almost every day” is filled in. 
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Example 3 


What do you think? Tell how much you agree with these 


statements. 
Fill in only one oval for each row. 
Agree Agree Disagree Disagree 
a lot a little a little a lot 

a) Watching movies is fun --------------- D <>) @ 

b) I like eating ice cream ‘o) @ @ 

c) I donot like waking up early -------- @D @ @D 

d) Ienjoy doing chores -------------------- D @ @ 


ima) Example 3 says, “What do you think? Tell how much you agree with these 
statements.” Letter (a) says, “Watching movies is fun.” You are given four choices 
for how much you agree with the statement: Agree a lot, Agree a little, Disagree a 
little, and Disagree a lot. 


(J) In this example, the student agrees a little that watching movies is fun and the oval 
under “Agree a little” is filled in. 


Give students time to read through the examples. Then continue reading the final directions: 
O Read each question carefully, and pick the answer you think is best. 
(0 Fill in the oval next to or under your answet. 


ine If you decide to change your answer, completely erase your first choice. Then, fill in 
the oval next to or under your new answer. 


Ask for help if you do not understand something or are not sure how to answer. 


BB 


Are there any questions before we start? 


If there are questions try to answer them the best you can. If there are no more questions, record the 
current time in cell (13a) of the Test Administration Form and say: 


(J Turn the page to the first question and begin answering this questionnaire. You will 
have 30 minutes to answer these questions. 


After 30 minutes are up, say: 


LY Please raise your hand if you have finished answering the questions. 
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If not all of the students raise their hands, allow for additional time and say: 


L =You will have more time to continue answering this questionnaire. If you have 
already finished all the questions, then you can use this time to review your answers. 
Once you have finished, please close your booklet and read quietly at your desk. 


Once all students have finished and have closed their booklets, record the current time in cell (13b) 
of the Test Administration Form. Then say: 


ina Thank you very much for participating in this study. Your work will help us to learn 
more about our students and schools. Please stay seated. 


ima Please close your booklets and insert the booklet into the envelope it was in when 
you received it and seal the envelope closed. Once you have done this, remove the 
label that contains your name from the envelope. Peel it off and I will collect it. 


Collect the labels tear them up and throw them away. Then, collect all test booklets and envelopes 
and keep them secure. Check against the Student Tracking Form to make sure that you have 
received all of them. Use the sequential number you assigned to the front of the envelope to help in 
this process. 


Thank you again for your help in conducting this important 
international study. 


Please answer questions 14 through 18 on the Test Administration Form. 
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TRENDS IN INTERNATIONAL 
MATHEMATICS AND SCIENCE 
STUDY —- ADVANCED 2015 
SESSION SCRIPT: GRADE 12 


Administering the TIMSS Advanced Assessment 


The instructions marked with the LJ symbol and printed in bold in the administration script must 
be read aloud to the students word for word to ensure that the testing sessions are conducted in the 
same way in all countries. Although you should become familiar with these instructions before the 
actual testing, do not attempt to memorize them. Read these instructions exactly as they ate written. 
Comments that are not in bold are not to be read aloud. They are instructions for you only. 


To begin the testing session: 
- Make sure that the Class ID is recorded at the top of the Test Administration Form. If 
missing, you can find it on the Student Tracking Form, column [Db]. 


- Make sure that students are seated quietly, with nothing on their desks except for a 
pencil and their own calculator. 


- Record the current time in cell Qa) of the Test Administration Form. 


- Begin reading the administration script. 
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Introduction 


ian Good morning/afternoon, everyone! My name is (YOUR NAME). This school has 
been chosen to take part in an important international project to study what students 
around the world know and can do in advanced mathematics and physics. Different 
countries from all over the world are taking part in this study. You will be taking an 
advanced mathematics or physics test, and answering a short questionnaire. Please 
listen carefully while I read the instructions for today’s test. 


ian If you still have any schoolbooks, papets, or other materials on your desk, please put 
them away. You may use a calculator. However, you may not use any device that can 
connect to the Internet. All electronic devices, such as cell phones, tablets, portable 
computers, and photo or video cameras, must be stored away for the duration of the 
test administration. You may have a book to read should you finish the test early. 


LO Now I will pass out the test booklets. Do not remove the booklet from the envelope 
until I tell you to do so. Each of you will receive one test booklet. Not all booklets are 
the same. 


Students should have only a pencil, a calculator, and his or her booklet for the duration of test 
administration. 


As you hand out the test booklets, make sure that each student receives the booklet specially 
prepared for him or her. You can do that by reading the name on each test booklet, and giving it to 
that student. Do not allow students to open the test booklets until you tell them to. 


Record the student’s participation status in column 7 (Achievement Session) of the Student Tracking 
Form. 


If you are administering a makeup session, use the shaded part of column 7 (Achievement Session). 


If a student is absent, put that test booklet aside. Do not give it to anyone else, since each test 
booklet is marked for a specific student. 


If there is a student in the classroom who is not listed on the Student Tracking Form, or an 
originally assigned booklet is damaged, assign one of the three spare booklets or one from your bulk 
supplies. 


After the test booklets are passed out and the Student Tracking Form has been completed, say the 
following to the students: 


ae) Does evetybody have his or her test booklet? 


If yes, then continue. If not, find out why and proceed as described before. 
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ina Please take your test booklet out of the envelope. Turn to the first page in the booklet 
that says “General Directions.” You will now have five minutes to read the directions 
quietly. 


Please do not admit any students into the testing session at this point. 


Below are the general directions that the students will read in their test booklets. 


General Directions 


You will have 90 minutes to answer the questions in this booklet. It is important that you attempt 
every question, giving the best answer that you can. 


Calculators may be used, and [mathematics/physics] formulas are provided on the pages following 
these directions. However, you may not use any device that can connect to the Internet. 


Some questions require you to select the correct answer from the options provided and fill in the 
oval next to the answer you believe to be correct, as shown in Example 1. 


Example 1 in Advanced Mathematics booklets: 


Example 1 
What is the formula for the area of a circle? 


® 


A 
A 
© A=4nar’ 
@© A 


Example 1 in Physics booklets: 


Example 1 
Which is a unit of energy? 


(@) newton 
kilogram 
© joule 

©) hertz 


If you are not sure of the answer to a question, fill in the answer you think is most likely to be 
correct. 


If you decide to change your answer, completely erase your first choice. Then, fill in the oval next to 
your new choice. 
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Other questions require you to write answers in the space provided in your booklet. Some of 
these questions require you to “show your work.” For these questions, be sure to give a complete 
explanation or show all the steps you used to obtain your answer. 


Example 2 in Advanced Mathematics booklets: 


Example 2 


What are the points of intersection of the functions f(x)=x+2 and g(x)= x? 


Show your work. 


x* = x#2 


HOS eS BZ =o 


Cx-2) Cx+l)=0 


X¥>+2 o0¢ xe7l 


If you use a calculator in answering one of these questions, you still must describe all the steps you 
used to obtain your answer. The following illustrates a calculator explanation of Example 2. 


Graened £cx)=X+*2 ond 


BX=x* on carculator 


INTERSECTION — x=2 and x= -1\ 


£c2z 24 
£(-1) = 4 
(2,4) and 1,1.) 
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Example 2 in Physics booklets: 


Example 2 


A 10 meter copper bar, with a coefficient of linear expansion of 17x10° / °C, is 
heated 100 °C. What is the new length of the bar? 


Show your work. 
KQ= d,a bT 
AL + (to) (itx™+) ( tee) 
AL a/R xlo "my 
{e+ Abe tot .ol7m = (0.0l7m 


If you use a calculator in answering one of these questions you still must describe all the steps you 
used to obtain your answer. The following illustrates a calculator explanation of Example 2. 


TRE STE py le the equa bib Nz = doo AT, I Used 
MY caleuletor te get AL, tha ladded that % J, 
bs qt (0.D17m 


When you have finished answering all of the questions in this booklet, please answer the short 
sutvey on calculator use at the end of the booklet. 


Instructions 


After the 5 minutes have passed (if it seems that students have not finished reading through the 
general directions, give them a few extra minutes), say: 


[ =Do you have any questions? 
If there are questions, try to answer them as best you can. If there are no more questions, say: 


ine Do your best to answer all the questions. If you cannot answer a question, move onto 
the next one. 


(2 “Please turn the page, and start working immediately. You have 90 minutes to 
complete the test and 5 minutes to complete the calculator use survey. You will then 
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have a short break. After the break, you will be asked to complete a short 
questionnaire. 


Record the current time in cells [9b] and [10a] of the Test Administration Form and begin timing the 
90 minutes allocated for the testing session. 


Remember, you are not allowed to help the students with the test. 
Ten minutes before the testing session ends, say the following: 


(Q You have 10 minutes of testing time left. Make sure you try to finish answering all of 
the questions in your booklet. 


After the last 10 minutes have passed, record the current time in cell [10b] of the Test 
Administration Form and say: 


ian Your time is up. Please turn to the next page following the assessment questions 
where you will find a short survey on calculator use. You have 5 minutes to answer 
the survey. 


Make sure that the students have their booklets open to the Calculator Use Survey page before 
proceeding. 


After the 5 minutes have passed, say: 


(J Your time is up. Please close your booklets and put your pencils down. Do not write 
anything more. We will now take a short break. Afterwards, I will ask you to answer a 
short questionnaire. If you leave the room, please be back on time. 


Break 


If the room will be left unattended during the break, collect the booklets in the envelopes from the 
students one by one. Keep the booklets secure during the break time. You will then redistribute the 
booklets after the break, just like you did at the beginning of the testing session, making sure that 
each student receives the same test booklet he/she was working on during the first half of the 
testing session. 


Administering the Student Questionnaire 


Before you begin the questionnaire administration: 


- Make sure that you have the corresponding Student Tracking Form and Test 
Administration Form. 


- Make sure all the students are in the class; all are seated quietly, and have a pencil. 
When ready, say: 


ian Now you will complete a short questionnaire. Turn to the colored divider page in 
your booklet. Do not begin the questionnaire until I tell you to do so. 
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CQ The directions are printed at the beginning of your questionnaire. It is important 
that you follow the directions very carefully so that you understand how to mark your 
answers. Now turn to the first page titled “Directions.” 


Make sure that the students have their questionnaires open to the Directions page before 


proceeding. 
CQ You will have 5 minutes to read the directions quietly. 


Below are the general directions that the students will read in their questionnaires. 


Directions 


In this booklet, you will find questions about yourself. Some questions ask for facts while other 
questions ask for your opinion. 


Each question is followed by a number of answers. Shade in the circle next to or under the answer 
of your choice as shown in the example below. 


Example 


How often do you do these things? 


Fill in only one oval for each row. 


Every day Once or Once or Never or 
or almost twice a twice a almost 
every day week month never 

a) I talk with my friends ------------------ @ >) @D 

b) I play sports ------------------------------- o) >) o>) 

c) I listen to music -------------------------- DO @ ®D 


e Read each question carefully, and pick the answer you think is best. 
e Fill in the oval next to or under your answer. 


e If you decide to change an answer, completely erase your first choice. Then, fill in the oval 
next to or under your new answer. 


e Ask for help if you do not understand something or are not sure how to answer. 
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Instructions 


After the 5 minutes have passed (if it seems that students have not finished reading through the 
general directions, give them some extra minutes), say: 


ae Do you have any questions? 


If there are questions, try to answer them as best you can. If there are no more questions, record the 
current time in cell [11a] of the Test Administration Form and say. 


ima Please begin answering this questionnaire. You will have 30 minutes to answer these 
questions. 


After 30 minutes are up, say: 
ian Please raise your hand if you have finished answering the questions. 
If not all of the students raise their hands, allow for additional time and say: 


(= =You will have mote time to continue answering this questionnaire. If you have 
already finished all the questions, then you can use this time either to review your 
answers or to read quietly at your desk. 


Once all students have finished and have closed their booklets, record the current time in cell [11b] 
in the Test Administration Form. Then say: 


ima Thank you very much for participating in this study. Your work will help us to learn 
more about our students and schools. Please stay seated. 


ian Please close your booklets and insert the booklet into the envelope it was in when 
you received it and seal the envelope closed. Once you have done this, remove the 
label that contains your name from the envelope. Peel it off and I will collect it. 


Collect the labels tear them up and throw them away. Then, collect all test booklets and envelopes 
and keep them secure. Check against the Student Tracking Form to make sure that you have 
received all of them. Use the sequential number you assigned to the front of the envelope to help in 
this process. 


Thank you again for your help in conducting this important 


international study. 


Please answer questions 12 through 16 on the Test Administration Form. 


U.S. TIMSS 2015 and TIMSS Advanced 1995 & 2015 Technical Report and User’s Guide C-38 


